-- Logs begin at Thu 2022-09-22 21:30:47 UTC, end at Thu 2022-09-22 22:37:38 UTC. -- Sep 22 21:55:21.914688 dv-cinder-849143-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 22 21:55:21.927636 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 22 21:55:21.928718 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 22 21:55:21 2022] *** Sep 22 21:55:21.928718 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 22 21:55:21.928718 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 22 21:55:21.928718 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: nodename: dv-cinder-849143-6-iscsi Sep 22 21:55:21.928718 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: machine: x86_64 Sep 22 21:55:21.928718 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: clock source: unix Sep 22 21:55:21.928718 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: pcre jit disabled Sep 22 21:55:21.928718 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: detected number of CPU cores: 4 Sep 22 21:55:21.928718 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: current working directory: / Sep 22 21:55:21.929538 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: detected binary path: /usr/bin/uwsgi-core Sep 22 21:55:21.929538 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: your processes number limit is 47680 Sep 22 21:55:21.929538 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: your memory page size is 4096 bytes Sep 22 21:55:21.929538 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: detected max file descriptor number: 2048 Sep 22 21:55:21.929538 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: lock engine: pthread robust mutexes Sep 22 21:55:21.929538 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: thunder lock: enabled Sep 22 21:55:21.929538 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 22 21:55:21.929538 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 22 21:55:21.968555 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: Python main interpreter initialized at 0x5641e66b29c0 Sep 22 21:55:21.968555 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: python threads support enabled Sep 22 21:55:21.968555 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: your server socket listen backlog is limited to 100 connections Sep 22 21:55:21.968555 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: your mercy for graceful operations on workers is 90 seconds Sep 22 21:55:21.968968 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: mapped 403077 bytes (393 KB) for 2 cores Sep 22 21:55:21.968968 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: *** Operational MODE: preforking *** Sep 22 21:55:21.969292 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: *** uWSGI is running in multiple interpreter mode *** Sep 22 21:55:21.969292 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: spawned uWSGI master process (pid: 81616) Sep 22 21:55:21.969504 dv-cinder-849143-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 22 21:55:21.976187 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: spawned uWSGI worker 1 (pid: 81628, cores: 1) Sep 22 21:55:21.976187 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: spawned uWSGI worker 2 (pid: 81629, cores: 1) Sep 22 21:55:21.976187 dv-cinder-849143-6-iscsi devstack@keystone.service[81616]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 22 21:55:26.281637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 21:55:26.281637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: from cryptography.utils import int_from_bytes Sep 22 21:55:26.282423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 21:55:26.282423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: from cryptography.utils import int_from_bytes Sep 22 21:55:26.282736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 21:55:26.282736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: from cryptography.utils import int_from_bytes Sep 22 21:55:26.283190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 22 21:55:26.283190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: from cryptography.utils import int_from_bytes Sep 22 21:55:26.332833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 21:55:26.333002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 21:55:26.333144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] command line args: [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 21:55:26.333416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 21:55:26.333503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 22 21:55:26.333583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ================================================================================ {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 21:55:26.333667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 22 21:55:26.333829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] admin_token = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.333909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] command line args: [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 22 21:55:26.334071 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] config_dir = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.334149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 22 21:55:26.334256 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.334337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ================================================================================ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 22 21:55:26.334435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] config_source = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.334519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] admin_token = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.334616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] debug = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.334753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] config_dir = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.334842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.335110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.335110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] config_source = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.335296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.335391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] debug = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.335499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.335603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.335735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.335840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.335942 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.336047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.336149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.336252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.336353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_config_append = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.336353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.336542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.336654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_dir = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.336751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.336846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_file = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.336941 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_config_append = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.337043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.337154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.337154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_dir = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.337349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.337424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.337539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.337619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.337697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.337827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.337903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.337903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.338094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.338166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.338240 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.338319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.338435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.338555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.338631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.338758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.338844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.338963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.339047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.339200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.339353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.339432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.339524 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.339659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.339737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.339895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.339979 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.340149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.340399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.340476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.340574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] publish_errors = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.340677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.340804 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.341035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.341194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.341278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.341358 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] publish_errors = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.341476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.341581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.341687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.341815 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.341911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] standard_threads = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.342036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.342117 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.342227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.342355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.342466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.342551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.342690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] use_journal = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.342767 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] standard_threads = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.342864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] use_json = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.342942 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.343065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] use_stderr = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.343145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.343217 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] use_syslog = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.343314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.343398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.343493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] use_journal = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.343651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.343731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] use_json = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.343833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] use_stderr = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.343909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.344011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] use_syslog = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.344133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.344243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 22 21:55:26.344372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.344503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.344619 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.344748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.344857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.345290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.345379 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.345536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.345615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.345816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.346478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.346675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.346748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.346928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.347106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.347180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.347250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.347382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.347466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.347543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.347647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.347786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.347900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.348031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.348141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.348268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.348394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.348521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.348634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.348772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.348886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.349016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.349159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.349275 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.349472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.349556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.349652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.349772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.349912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.350022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.350162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.350269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.350401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.350501 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.350658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.350736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.350922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.351001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.351176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.351253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.351422 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.351500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.351656 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.351734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.351885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.351961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.352114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.352190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.352348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.352431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.352591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.352667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.352829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.352906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.353061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.353166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.353318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.353438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.353553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.353637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.353786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.353872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.354016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.354094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.354243 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.354320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.354472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.354551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.354701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.354779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.354935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.355020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.355168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.355247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.355389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.355468 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.355622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.355707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.355852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.355931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.356086 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.356164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.356318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.356395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.356555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.356633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.356784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.356897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.357020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.357144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.357280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.357391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.357517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.357620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.357750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.357863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.358016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.358157 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.358274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.358384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.358468 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.358588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.358710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.358792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.358909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.358985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.359100 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.359184 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.359343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.359457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.359582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.359658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.359795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.359923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.360035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.360125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.360268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.360348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.360532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.360615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.360767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.360852 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.361001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.361171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.361261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.361348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.361518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.361658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.361857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.361932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.362078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.362186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.362364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.362458 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.362606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.362685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.362846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.362926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.363083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.363169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.363319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.363397 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.363567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.363650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.363803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.363888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.364036 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.364115 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.364273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.364352 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.364510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.364603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.364744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.364823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.364984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.365123 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.365249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.365341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.365492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.365571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.365729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.365806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.365967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.366053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.366208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.366285 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.366446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.366524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.366682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.366836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.366911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.366981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.367136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.367214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.367367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.367455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.367597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.367677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.367831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.367910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.368060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.368138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.368292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.368370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.368525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.368602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.368756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.368833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.368996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.369076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.369256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.369338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.369519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.369600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.369744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.369824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.369973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.external = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.370050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.370203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.370293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.370474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.370562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.370719 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.370798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.370955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.password = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.371034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.371184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] auth.token = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.371263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.external = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.371416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.371495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.371648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.371726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.371893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.371973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.372123 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.372204 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.password = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.372368 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.372448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] auth.token = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.372610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.372688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.372838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.372917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.373073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.373212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.373332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.373413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.373579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.373659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.373826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.373906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.374058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.374136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.374298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.374379 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.374530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.374608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.374775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.374855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.375023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.375103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.375255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.375333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.375506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.375584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.375751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.375832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.376000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.376080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.376257 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.376343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.376495 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.376575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.376744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.376823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.376979 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.377129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.377281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.377421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.377562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.377680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.377826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.377916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.378064 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.378142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.378308 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.378386 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.378542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.378628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.378790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.378869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.379024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.379101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.379263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.379356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.379505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.379584 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.379742 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.379830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.379975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.380062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.380221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.380301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.380465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.380545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.380699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.380856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.380933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.381005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.381193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.381279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.381424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.381520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.381661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.381741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.381906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.381990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.382145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.382232 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.382371 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.382449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.382593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.382678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.382815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.382903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.383055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.383137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.383344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.383423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.383590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.383678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.383835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.383920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.384073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.384315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.384395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.384564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.384784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.384864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.385061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.385255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.385335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.385596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.385677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.385916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.386099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.386185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.386401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.386551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.386632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.386869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.386991 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.387111 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.387357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.387443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.387601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.387854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.387932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.388091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.388306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.388394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.388579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.388777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.388857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.389055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.389259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.389342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.389553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.389727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.389807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.390020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.390199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.390281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.390494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.390649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.390729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.390967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.391133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.391214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.391448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.391611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.391691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.391921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.392043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.392156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.392506 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.392586 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.393376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.393517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.393626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.393871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.393949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.394116 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.394356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.394435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.394604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.394686 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.394871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.394951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.395111 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.395343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.395431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.395608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.395725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.395861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.396098 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.396178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.396339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.396418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.396576 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.396654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.396813 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.396891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.397057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.397191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.397317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.397397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.397563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.397641 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.397806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.397887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.398042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.398138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.398287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.398369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.398535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.398615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.398770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.398849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.399023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.399102 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.399267 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.399400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.399508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.399751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.399829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.399993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.400245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.400326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.400498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.400660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.400740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.401057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.401165 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.401321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.401572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.401650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.401815 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.402030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.402119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.402283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.402475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.402555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.402767 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.402946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.403034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.403232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.403412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.403491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.403715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.403892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.403980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.404199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.404369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.404449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.404693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.404830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.404922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.405192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.405305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.405437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.405682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.405790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.405921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] role.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.406182 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] role.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.406260 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.406418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.406672 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.406750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.406926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.407123 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.407203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.407416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.407587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.407675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.407903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.408063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.408143 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.408944 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.409217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.409317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.409478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.409603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.409730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.409885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.409979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.410220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.410311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.410495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.410628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.410743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.410868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.410988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.411122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.411235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.411377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.411497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.411630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.411745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.411885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.411965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.412124 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.412204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.412384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.412477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.412641 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.412722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.412885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.412965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.413141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.413239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.413382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.413463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.413620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.413700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.413869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.413957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.414120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.414201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.414361 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.414442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.414599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.414687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.414834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] role.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.414913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.415064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.415143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] role.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.415303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.415390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.415543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.415622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.415791 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.415871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.416029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.416116 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.416275 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.416355 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.416521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.416602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.416760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.416840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.417003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.417081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.417273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.417352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.417504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.417590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.417752 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.417833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.417992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.418070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.418231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.418313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.418472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.418551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.418753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.418875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.418990 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.419117 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.419237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.419369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.419504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.419621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.419768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.419872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.420045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.420185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.420317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.420445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.420575 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.420712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.420841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.420980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.421127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.421253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.421381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.421519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.421647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.421771 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.421896 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.422023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.422151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.422286 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.422414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.422541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.422675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.422801 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.422928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.423065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.423192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.423318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.423446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.423573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.423701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.423841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.424002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.424133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.424261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.424388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.424594 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.424736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.424864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81629) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 21:55:26.424990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.424990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.424990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.424990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.425348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.425466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.425707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.425945 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.426180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.426427 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.426709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.426941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.427184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.427447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.427710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.connection = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.427957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.428207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.428447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.428688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.428925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.429185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.429425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.429662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.429910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.430148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.430395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.430630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.430877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.431387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.431642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.431874 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.432113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.432348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 22 21:55:26.432517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81628) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 22 21:55:26.453661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.453892 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.454119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.457729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.457958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.458203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.463162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.466393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.492395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.492629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.492799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.493031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.546410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.547076 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.595722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.595927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.596479 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.596716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.751399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.751629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.751855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.751941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.752109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.752262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.752398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.752504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.752582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.752800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.752887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.753097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.775415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.775651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.775736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.775882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.775965 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.776106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.776188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.776324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.776415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.776539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.776621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.776747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.776828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.776959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.788603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.788727 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.802035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.802154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.802251 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.802377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.802461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.802607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.802690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.802833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.802926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.803063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.803190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.803276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81628) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.803353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.803485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81629) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 22 21:55:26.819849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.820054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 21:55:26.820393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.820591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 21:55:26.820674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.820854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 21:55:26.820937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.821132 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.821216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.821402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.821498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 22 21:55:26.821669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.821819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.821930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.822194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.822455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.831254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.831530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.831799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.832080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.832314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.832599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.832875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.833200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.839283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.840154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.842181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.842466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.842743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.843020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.843101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.843299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.843433 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.843558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.843641 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.843824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.843907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.844092 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.844174 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.844357 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.844438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.844626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.844715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.844893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.845250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.845481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.846318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.846564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.868557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.869826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.870029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.870313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.871291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.871580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.874922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.875213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.875493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.876251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.876542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.876827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.882138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.882425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.883271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.883562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.887979 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.888272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.888552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.888833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.889184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.889325 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.889458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.889590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.889733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.889863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.890001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.890130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.890311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.890589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.890863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.891133 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.905515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.905787 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.906626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.906910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.910135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.911491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.913363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.913657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.914771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.915070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.917849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.919279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.921288 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.922480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.924561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.924858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.925163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.925455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.925733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.926010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.926223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.926530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.926819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.927106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.927398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.927697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.942440 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.942726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.942994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.943258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.944116 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.944407 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.944675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.944939 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.950066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.951598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.951809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.953124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.953385 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.953512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.954918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.955307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.956229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.956511 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.956788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.957067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.957484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.958120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.958409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.958694 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.958974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.959367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.968498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.968780 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.969059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.969366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.969743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.970274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.970559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.970843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.971178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.971470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.971752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.972134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.972421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.974987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.986876 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.989970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.990944 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.992819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.993904 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.994290 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.995745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:26.996023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.996419 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.996794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.998287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:26.999775 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:27.000063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.000476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.003491 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.004946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:27.007425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.007839 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.008231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.008516 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.008793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.008930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:27.011796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.012199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.012492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.012772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.016528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:27.016822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.017099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.020711 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:27.021008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.021331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.024215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81628) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:27.024506 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.024785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.025060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.025366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.025637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.025909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.026175 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.026459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.026723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.027011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.027319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.027656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.027942 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81628) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.028215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81629) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 22 21:55:27.028505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.028805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.029088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.029401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.029678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.029954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.030235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.030508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.030788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.031077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.031389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.031701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.031992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81629) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 22 21:55:27.062488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5641e66b29c0 pid: 81628 (default app) Sep 22 21:55:27.066536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805231-88a6-4414-b256-89bfa6faf890 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:27.066822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805231-88a6-4414-b256-89bfa6faf890 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:27.066952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5641e66b29c0 pid: 81629 (default app) Sep 22 21:55:27.067094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805231-88a6-4414-b256-89bfa6faf890 None None] PATH_INFO: `/v3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:27.071255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Thu Sep 22 21:55:27 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:33.887824 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd139ee-8053-437a-a1bd-9a8c651f3d8f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:33.888362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd139ee-8053-437a-a1bd-9a8c651f3d8f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:33.888362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd139ee-8053-437a-a1bd-9a8c651f3d8f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:33.893081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:55:33.898442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e965b04-091d-40b2-8912-ed4f9f6b684b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:33.898976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e965b04-091d-40b2-8912-ed4f9f6b684b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:33.898976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e965b04-091d-40b2-8912-ed4f9f6b684b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:33.968794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: INFO dbcounter [None req-0e965b04-091d-40b2-8912-ed4f9f6b684b None None] Registered counter for database keystone Sep 22 21:55:33.988586 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG oslo_db.sqlalchemy.engines [None req-0e965b04-091d-40b2-8912-ed4f9f6b684b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81628) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 21:55:33.994725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writer thread running {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 21:55:34.172523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG passlib.handlers.bcrypt [None req-0e965b04-091d-40b2-8912-ed4f9f6b684b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81628) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 21:55:34.172840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG passlib.handlers.bcrypt [None req-0e965b04-091d-40b2-8912-ed4f9f6b684b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81628) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 21:55:34.231128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0e965b04-091d-40b2-8912-ed4f9f6b684b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:34.232732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0e965b04-091d-40b2-8912-ed4f9f6b684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:34.359083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 22 21:55:34.817405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd139ee-8053-437a-a1bd-9a8c651f3d8f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:34.886503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: INFO dbcounter [None req-8fd139ee-8053-437a-a1bd-9a8c651f3d8f None None] Registered counter for database keystone Sep 22 21:55:34.907315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG oslo_db.sqlalchemy.engines [None req-8fd139ee-8053-437a-a1bd-9a8c651f3d8f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81629) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 22 21:55:34.914251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writer thread running {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 22 21:55:35.032097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd139ee-8053-437a-a1bd-9a8c651f3d8f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:35.037012 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727080cc-2eac-4d75-8499-1bbf3132eea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:35.037678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727080cc-2eac-4d75-8499-1bbf3132eea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:35.037958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727080cc-2eac-4d75-8499-1bbf3132eea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:35.038226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727080cc-2eac-4d75-8499-1bbf3132eea4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:35.046395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727080cc-2eac-4d75-8499-1bbf3132eea4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:35.098621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: WARNING keystone.common.rbac_enforcer.enforcer [None req-727080cc-2eac-4d75-8499-1bbf3132eea4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 21:55:35.098905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727080cc-2eac-4d75-8499-1bbf3132eea4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:35.108552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Sep 22 21:55:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 21:55:35.115107 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e965b04-091d-40b2-8912-ed4f9f6b684b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:35.132242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e965b04-091d-40b2-8912-ed4f9f6b684b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:35.136625 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cfe315-0e42-4438-aad9-3c9ecf557ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:35.137801 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfe315-0e42-4438-aad9-3c9ecf557ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:35.138101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfe315-0e42-4438-aad9-3c9ecf557ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:35.138366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cfe315-0e42-4438-aad9-3c9ecf557ca7 admin admin] PATH_INFO: `/v3/projects/f680186a14a640efb3f020f909ac5fa9` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:35.139924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfe315-0e42-4438-aad9-3c9ecf557ca7 admin admin] RBAC: Authorizing `identity:get_project(project_id=f680186a14a640efb3f020f909ac5fa9)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:35.192616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: WARNING keystone.common.rbac_enforcer.enforcer [None req-70cfe315-0e42-4438-aad9-3c9ecf557ca7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 22 21:55:35.192959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cfe315-0e42-4438-aad9-3c9ecf557ca7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:35.197466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Thu Sep 22 21:55:35 2022] GET /identity/v3/projects/f680186a14a640efb3f020f909ac5fa9 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:37.328775 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c66084-d9bf-425a-893a-51d800ac80c2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:37.329293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c66084-d9bf-425a-893a-51d800ac80c2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:37.329293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c66084-d9bf-425a-893a-51d800ac80c2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:37.333387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:37.338909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa848-cfed-4170-bfe2-10d1910860f8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:37.339288 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa848-cfed-4170-bfe2-10d1910860f8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:37.339467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa848-cfed-4170-bfe2-10d1910860f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:37.359814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8abaa848-cfed-4170-bfe2-10d1910860f8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:37.362028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8abaa848-cfed-4170-bfe2-10d1910860f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:37.369812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:37 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 22 21:55:37.637469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c66084-d9bf-425a-893a-51d800ac80c2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:37.649972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c66084-d9bf-425a-893a-51d800ac80c2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:37.654221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81963e00-69ad-4781-81ed-8a8e934b7f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:37.654814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81963e00-69ad-4781-81ed-8a8e934b7f1f admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:37.655094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81963e00-69ad-4781-81ed-8a8e934b7f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:37.655368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81963e00-69ad-4781-81ed-8a8e934b7f1f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:37.663528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81963e00-69ad-4781-81ed-8a8e934b7f1f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:37.665304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81963e00-69ad-4781-81ed-8a8e934b7f1f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:37.673994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Thu Sep 22 21:55:37 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 21:55:37.679348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa848-cfed-4170-bfe2-10d1910860f8 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:37.688267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa848-cfed-4170-bfe2-10d1910860f8 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:37.692544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17392b9d-7341-470f-b861-0002dc112c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:37.693187 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17392b9d-7341-470f-b861-0002dc112c8b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:37.693465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17392b9d-7341-470f-b861-0002dc112c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:37.693733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17392b9d-7341-470f-b861-0002dc112c8b admin admin] PATH_INFO: `/v3/users/9a8dd819f6d742ceba5a020c855827f8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:37.695026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17392b9d-7341-470f-b861-0002dc112c8b admin admin] RBAC: Authorizing `identity:get_user(user_id=9a8dd819f6d742ceba5a020c855827f8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:37.696804 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17392b9d-7341-470f-b861-0002dc112c8b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:37.698368 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Thu Sep 22 21:55:37 2022] GET /identity/v3/users/9a8dd819f6d742ceba5a020c855827f8 => generated 247 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:39.983638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a769e8d-fab0-4908-8520-641c28f0a7da None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:39.984239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a769e8d-fab0-4908-8520-641c28f0a7da None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:39.984239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a769e8d-fab0-4908-8520-641c28f0a7da None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:39.986045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:39.991191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f076f0-1f2d-4edc-a4fb-451c802241c4 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:39.991592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f076f0-1f2d-4edc-a4fb-451c802241c4 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:39.991728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f076f0-1f2d-4edc-a4fb-451c802241c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:40.010266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-12f076f0-1f2d-4edc-a4fb-451c802241c4 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:40.011446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-12f076f0-1f2d-4edc-a4fb-451c802241c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:40.101788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 6/11] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:40.147151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be300a-9abb-46ef-9701-03eb4b58db6d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:40.147435 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be300a-9abb-46ef-9701-03eb4b58db6d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:40.147708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09be300a-9abb-46ef-9701-03eb4b58db6d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:40.149033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 6/12] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:40.154253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428a900-5cd1-45a6-a84b-3cd3386cd365 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:40.154547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428a900-5cd1-45a6-a84b-3cd3386cd365 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:40.154825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7428a900-5cd1-45a6-a84b-3cd3386cd365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:40.173606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7428a900-5cd1-45a6-a84b-3cd3386cd365 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:40.174806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7428a900-5cd1-45a6-a84b-3cd3386cd365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:40.191734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:40.244772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:40.365550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be300a-9abb-46ef-9701-03eb4b58db6d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:40.509803 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09be300a-9abb-46ef-9701-03eb4b58db6d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:40.512398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428a900-5cd1-45a6-a84b-3cd3386cd365 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:40.543687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8617a96-468f-4811-9e29-7d8fb45f215d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:40.544541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8617a96-468f-4811-9e29-7d8fb45f215d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:40.544819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8617a96-468f-4811-9e29-7d8fb45f215d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:40.545090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8617a96-468f-4811-9e29-7d8fb45f215d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:40.557091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8617a96-468f-4811-9e29-7d8fb45f215d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b8617a96-468f-4811-9e29-7d8fb45f215d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:55:40.568258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:55:40.569221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:55:40.570692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 7/14] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:55:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:55:40.575060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8617a96-468f-4811-9e29-7d8fb45f215d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:40.608270 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:40.612893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7428a900-5cd1-45a6-a84b-3cd3386cd365 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:40.647266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50291023-420c-4957-b63f-f8b1c5c240be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:40.647881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50291023-420c-4957-b63f-f8b1c5c240be None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:40.648148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50291023-420c-4957-b63f-f8b1c5c240be None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:40.648418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50291023-420c-4957-b63f-f8b1c5c240be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:40.669011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:40.678313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8617a96-468f-4811-9e29-7d8fb45f215d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:40.711668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb185f-2631-4152-bdcc-b3dfab8b424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:40.712273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb185f-2631-4152-bdcc-b3dfab8b424c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:40.712549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb185f-2631-4152-bdcc-b3dfab8b424c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:40.712818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb185f-2631-4152-bdcc-b3dfab8b424c None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:40.713508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb185f-2631-4152-bdcc-b3dfab8b424c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:40.713805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb185f-2631-4152-bdcc-b3dfab8b424c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:40.715186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cb185f-2631-4152-bdcc-b3dfab8b424c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:40.716295 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50291023-420c-4957-b63f-f8b1c5c240be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:40.717788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50291023-420c-4957-b63f-f8b1c5c240be None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:40.727497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 8/15] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:55:40.732935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb185f-2631-4152-bdcc-b3dfab8b424c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:40.815838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 8/16] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:55:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:55:40.821439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50291023-420c-4957-b63f-f8b1c5c240be None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:40.824932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:40.830284 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cb185f-2631-4152-bdcc-b3dfab8b424c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:40.864763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ba0a6-ee47-4e6f-81ac-66b7c39f775f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:40.865413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba0a6-ee47-4e6f-81ac-66b7c39f775f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:40.865692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba0a6-ee47-4e6f-81ac-66b7c39f775f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:40.865963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ba0a6-ee47-4e6f-81ac-66b7c39f775f None admin] PATH_INFO: `/v3/users/9a8dd819f6d742ceba5a020c855827f8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:40.867504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ba0a6-ee47-4e6f-81ac-66b7c39f775f None admin] RBAC: Authorizing `identity:get_user(user_id=9a8dd819f6d742ceba5a020c855827f8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:40.868987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ba0a6-ee47-4e6f-81ac-66b7c39f775f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:40.870554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 9/17] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:55:40 2022] GET /identity/v3/users/9a8dd819f6d742ceba5a020c855827f8 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:55:40.872527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:40.876326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ba0a6-ee47-4e6f-81ac-66b7c39f775f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:40.922353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50291023-420c-4957-b63f-f8b1c5c240be None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:40.954918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27aa5a-5d60-44d3-98f6-05eccb3b1d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:40.955546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27aa5a-5d60-44d3-98f6-05eccb3b1d02 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:40.955824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27aa5a-5d60-44d3-98f6-05eccb3b1d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:40.956097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27aa5a-5d60-44d3-98f6-05eccb3b1d02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:40.966369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27aa5a-5d60-44d3-98f6-05eccb3b1d02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:40.967825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27aa5a-5d60-44d3-98f6-05eccb3b1d02 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:40.973230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ba0a6-ee47-4e6f-81ac-66b7c39f775f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-fa27aa5a-5d60-44d3-98f6-05eccb3b1d02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:55:40.986633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:55:40.987518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:55:40.988446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:55:40.988446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:55:40.988446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:55:40.988446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 9/18] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:55:40.993550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27aa5a-5d60-44d3-98f6-05eccb3b1d02 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:41.009977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148473d8-18c0-4b32-9d3b-e723450fab39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:41.010595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148473d8-18c0-4b32-9d3b-e723450fab39 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:41.010875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148473d8-18c0-4b32-9d3b-e723450fab39 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:41.011141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148473d8-18c0-4b32-9d3b-e723450fab39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:41.012318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148473d8-18c0-4b32-9d3b-e723450fab39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:41.013796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148473d8-18c0-4b32-9d3b-e723450fab39 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:41.015359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 10/19] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:55:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:41.020981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148473d8-18c0-4b32-9d3b-e723450fab39 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:41.055515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:41.098583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27aa5a-5d60-44d3-98f6-05eccb3b1d02 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:41.100759 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:41.122354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148473d8-18c0-4b32-9d3b-e723450fab39 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:41.132275 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3432c-b1cf-4c8c-999b-f28d1893dd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:41.132878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3432c-b1cf-4c8c-999b-f28d1893dd1f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:41.133177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3432c-b1cf-4c8c-999b-f28d1893dd1f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:41.133444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3432c-b1cf-4c8c-999b-f28d1893dd1f None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:41.134071 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3432c-b1cf-4c8c-999b-f28d1893dd1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:41.134369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3432c-b1cf-4c8c-999b-f28d1893dd1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:41.135736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3432c-b1cf-4c8c-999b-f28d1893dd1f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:41.152644 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 10/20] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:55:41.156535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6d0e1f-9fe5-4faa-897f-b550980037ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:41.157205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0e1f-9fe5-4faa-897f-b550980037ef None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:41.158036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0e1f-9fe5-4faa-897f-b550980037ef None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:41.158308 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6d0e1f-9fe5-4faa-897f-b550980037ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:41.159055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d0e1f-9fe5-4faa-897f-b550980037ef None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.domain.id=default, user.id=9a8dd819f6d742ceba5a020c855827f8) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:41.159357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d0e1f-9fe5-4faa-897f-b550980037ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:41.160833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6d0e1f-9fe5-4faa-897f-b550980037ef None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:41.170640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 11/21] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Sep 22 21:55:41 2022] GET /identity/v3/role_assignments?user.id=9a8dd819f6d742ceba5a020c855827f8&scope.domain.id=default&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:55:43.574237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15e493-904a-4448-991d-ace1c6a6fa52 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:43.574811 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15e493-904a-4448-991d-ace1c6a6fa52 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:43.574811 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15e493-904a-4448-991d-ace1c6a6fa52 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:43.576152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 11/22] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:43.595303 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45457650-22f1-4dad-bdeb-d406c893bd4e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:43.596267 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45457650-22f1-4dad-bdeb-d406c893bd4e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:43.596805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45457650-22f1-4dad-bdeb-d406c893bd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:43.619685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG passlib.handlers.bcrypt [None req-45457650-22f1-4dad-bdeb-d406c893bd4e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81629) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 22 21:55:43.620379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG passlib.handlers.bcrypt [None req-45457650-22f1-4dad-bdeb-d406c893bd4e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81629) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 22 21:55:43.678241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-45457650-22f1-4dad-bdeb-d406c893bd4e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:43.680136 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-45457650-22f1-4dad-bdeb-d406c893bd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:43.781251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 12/23] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:43.786199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e1a1f-9157-4118-b296-9923a5c7b265 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:43.786481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e1a1f-9157-4118-b296-9923a5c7b265 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:43.786753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e1a1f-9157-4118-b296-9923a5c7b265 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:43.788315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 12/24] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:43.803053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5e024-02b8-45b2-bf86-b307e9762675 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:43.803567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5e024-02b8-45b2-bf86-b307e9762675 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:43.804051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d5e024-02b8-45b2-bf86-b307e9762675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:43.825707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a7d5e024-02b8-45b2-bf86-b307e9762675 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:43.827161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a7d5e024-02b8-45b2-bf86-b307e9762675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:43.849582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:43.905069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 13/25] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:44.170726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcbe65-9671-4470-90d4-45e6f2c71556 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:44.171355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcbe65-9671-4470-90d4-45e6f2c71556 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:44.171830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dcbe65-9671-4470-90d4-45e6f2c71556 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:44.174472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 13/26] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:44.180223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc601b3e-7412-44e8-9002-dd6c22c843cb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:44.180742 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc601b3e-7412-44e8-9002-dd6c22c843cb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:44.181265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc601b3e-7412-44e8-9002-dd6c22c843cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:44.200107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cc601b3e-7412-44e8-9002-dd6c22c843cb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:44.201698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cc601b3e-7412-44e8-9002-dd6c22c843cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:44.284739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 14/27] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:55:44.323976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcbe65-9671-4470-90d4-45e6f2c71556 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:44.385145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc601b3e-7412-44e8-9002-dd6c22c843cb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:44.512554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dcbe65-9671-4470-90d4-45e6f2c71556 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:44.537502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:44.571296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc601b3e-7412-44e8-9002-dd6c22c843cb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:44.597393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8379f9e6-cdb5-4050-a292-3a3ee671c9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:44.598710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f9e6-cdb5-4050-a292-3a3ee671c9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:44.599226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f9e6-cdb5-4050-a292-3a3ee671c9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:44.599707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8379f9e6-cdb5-4050-a292-3a3ee671c9b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:44.620836 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379f9e6-cdb5-4050-a292-3a3ee671c9b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:44.622839 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8379f9e6-cdb5-4050-a292-3a3ee671c9b7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-8379f9e6-cdb5-4050-a292-3a3ee671c9b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:55:44.649147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:55:44.650048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:55:44.650846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:55:44.650846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:55:44.650846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:55:44.652141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fa5e8-e67d-444c-9a8b-18f7a0a8ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:44.653436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa5e8-e67d-444c-9a8b-18f7a0a8ca1f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:44.653911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa5e8-e67d-444c-9a8b-18f7a0a8ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:44.654376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fa5e8-e67d-444c-9a8b-18f7a0a8ca1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:44.654494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 14/28] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:55:44.655594 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4eacc-152c-4125-b5c4-940d8549cd64 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:44.655879 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4eacc-152c-4125-b5c4-940d8549cd64 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:44.656154 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4eacc-152c-4125-b5c4-940d8549cd64 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:44.658484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 15/29] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:44.664184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea2311-f529-414f-b148-4336410915f1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:44.664541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea2311-f529-414f-b148-4336410915f1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:44.664821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caea2311-f529-414f-b148-4336410915f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:44.669311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fa5e8-e67d-444c-9a8b-18f7a0a8ca1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-762fa5e8-e67d-444c-9a8b-18f7a0a8ca1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:55:44.676097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:55:44.676932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:55:44.677795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:55:44.680206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 15/30] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:55:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:55:44.681903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fa5e8-e67d-444c-9a8b-18f7a0a8ca1f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:44.684687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-caea2311-f529-414f-b148-4336410915f1 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:44.687192 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-caea2311-f529-414f-b148-4336410915f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:44.700692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:44.771819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 16/31] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:44.772362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea2311-f529-414f-b148-4336410915f1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:44.818848 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fa5e8-e67d-444c-9a8b-18f7a0a8ca1f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:44.860230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94135c49-142c-4124-baff-ebc44625c3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:44.861539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94135c49-142c-4124-baff-ebc44625c3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:44.862755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94135c49-142c-4124-baff-ebc44625c3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:44.863393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94135c49-142c-4124-baff-ebc44625c3f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:44.865562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94135c49-142c-4124-baff-ebc44625c3f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:44.866258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94135c49-142c-4124-baff-ebc44625c3f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:44.868844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94135c49-142c-4124-baff-ebc44625c3f9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:44.915693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 16/32] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:44.918167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94135c49-142c-4124-baff-ebc44625c3f9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:44.935757 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:44.976793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caea2311-f529-414f-b148-4336410915f1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:45.048287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2306c-f9e2-4081-bebd-f85105011ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:45.048822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2306c-f9e2-4081-bebd-f85105011ba3 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:45.049091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2306c-f9e2-4081-bebd-f85105011ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:45.049389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2306c-f9e2-4081-bebd-f85105011ba3 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:45.050152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2306c-f9e2-4081-bebd-f85105011ba3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:45.053157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2306c-f9e2-4081-bebd-f85105011ba3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:45.103780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94135c49-142c-4124-baff-ebc44625c3f9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:45.140280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926d64b-cd81-48c6-9648-b84b35d7be4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:45.141026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926d64b-cd81-48c6-9648-b84b35d7be4b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:45.141956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926d64b-cd81-48c6-9648-b84b35d7be4b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:45.142303 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926d64b-cd81-48c6-9648-b84b35d7be4b None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:45.143067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926d64b-cd81-48c6-9648-b84b35d7be4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:45.143469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926d64b-cd81-48c6-9648-b84b35d7be4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:45.144981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1926d64b-cd81-48c6-9648-b84b35d7be4b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:45.155645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 17/33] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:55:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:45.156537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926d64b-cd81-48c6-9648-b84b35d7be4b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:45.199361 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 17/34] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 22 21:55:44 2022] POST /identity/v3/roles => generated 219 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:55:45.205323 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2306c-f9e2-4081-bebd-f85105011ba3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:45.224901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:45.233789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:45.290755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926d64b-cd81-48c6-9648-b84b35d7be4b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:45.340591 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584d5c05-9029-4cb5-a879-bfd370577e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:45.341855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d5c05-9029-4cb5-a879-bfd370577e03 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:45.343883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d5c05-9029-4cb5-a879-bfd370577e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:45.344444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584d5c05-9029-4cb5-a879-bfd370577e03 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:45.345398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d5c05-9029-4cb5-a879-bfd370577e03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:45.347140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584d5c05-9029-4cb5-a879-bfd370577e03 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:45.372029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2306c-f9e2-4081-bebd-f85105011ba3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:45.421533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b54860-f5ea-428d-a0d9-e9dd0961621a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:45.422585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54860-f5ea-428d-a0d9-e9dd0961621a None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:45.423362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54860-f5ea-428d-a0d9-e9dd0961621a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:45.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b54860-f5ea-428d-a0d9-e9dd0961621a None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:45.424531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b54860-f5ea-428d-a0d9-e9dd0961621a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:45.426117 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b54860-f5ea-428d-a0d9-e9dd0961621a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:45.444269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 18/35] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 22 21:55:45 2022] POST /identity/v3/roles => generated 225 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:55:45.444897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b54860-f5ea-428d-a0d9-e9dd0961621a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:45.506120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 18/36] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:55:45 2022] POST /identity/v3/projects => generated 301 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:55:45.507741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f6467-60ca-497e-9478-2c152f890150 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:45.508002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f6467-60ca-497e-9478-2c152f890150 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:45.508284 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f6467-60ca-497e-9478-2c152f890150 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:45.509750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 19/37] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:45.516053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49286b-ce4a-4d41-a1e3-15b68c6277d3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:45.516536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49286b-ce4a-4d41-a1e3-15b68c6277d3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:45.517231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb49286b-ce4a-4d41-a1e3-15b68c6277d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:45.540643 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cb49286b-ce4a-4d41-a1e3-15b68c6277d3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:45.541613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:45.542791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cb49286b-ce4a-4d41-a1e3-15b68c6277d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:45.588431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b54860-f5ea-428d-a0d9-e9dd0961621a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:45.640555 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780849ee-509f-4940-a4ae-385b61155e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:45.643088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780849ee-509f-4940-a4ae-385b61155e5c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:45.643637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780849ee-509f-4940-a4ae-385b61155e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:45.645044 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780849ee-509f-4940-a4ae-385b61155e5c None admin] PATH_INFO: `/v3/users/9a8dd819f6d742ceba5a020c855827f8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:45.647366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780849ee-509f-4940-a4ae-385b61155e5c None admin] RBAC: Authorizing `identity:get_user(user_id=9a8dd819f6d742ceba5a020c855827f8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:45.649195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780849ee-509f-4940-a4ae-385b61155e5c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:45.651073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 19/38] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:55:45 2022] GET /identity/v3/users/9a8dd819f6d742ceba5a020c855827f8 => generated 247 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:45.652540 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f058d-374d-4691-85cf-bae27899a9f8 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:45.653054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f058d-374d-4691-85cf-bae27899a9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:45.653513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f058d-374d-4691-85cf-bae27899a9f8 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:45.654981 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 20/39] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:45.664756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f058d-374d-4691-85cf-bae27899a9f8 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:45.680369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 20/40] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:55:45.681813 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f41bf-02d2-4e84-b4b7-ae50f4a67101 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:45.682173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f41bf-02d2-4e84-b4b7-ae50f4a67101 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:45.682457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4f41bf-02d2-4e84-b4b7-ae50f4a67101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:45.710582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0c4f41bf-02d2-4e84-b4b7-ae50f4a67101 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:45.711805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0c4f41bf-02d2-4e84-b4b7-ae50f4a67101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:45.773263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:45.805170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 21/41] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:45.920419 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f058d-374d-4691-85cf-bae27899a9f8 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:45.955549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215f606-580f-4e1f-9ca3-4fbac3217418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:45.956170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f606-580f-4e1f-9ca3-4fbac3217418 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:45.956496 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f606-580f-4e1f-9ca3-4fbac3217418 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:45.956773 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f606-580f-4e1f-9ca3-4fbac3217418 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:45.957995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215f606-580f-4e1f-9ca3-4fbac3217418 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:45.959412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215f606-580f-4e1f-9ca3-4fbac3217418 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:45.961157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 21/42] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:55:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:45.966613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f41bf-02d2-4e84-b4b7-ae50f4a67101 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:45.982005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215f606-580f-4e1f-9ca3-4fbac3217418 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:45.985056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:46.077134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4f41bf-02d2-4e84-b4b7-ae50f4a67101 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:46.090913 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215f606-580f-4e1f-9ca3-4fbac3217418 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:46.112152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8c534-b683-40db-afaa-63f4510abd8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:46.112817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8c534-b683-40db-afaa-63f4510abd8a None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:46.113096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8c534-b683-40db-afaa-63f4510abd8a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:46.113408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8c534-b683-40db-afaa-63f4510abd8a None admin] PATH_INFO: `/v3/domains/default/users/9a8dd819f6d742ceba5a020c855827f8/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:46.115590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8c534-b683-40db-afaa-63f4510abd8a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9a8dd819f6d742ceba5a020c855827f8, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:46.117198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8c534-b683-40db-afaa-63f4510abd8a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:46.126525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d4b2-3f6c-4e65-b728-7722e935aa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:46.127276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d4b2-3f6c-4e65-b728-7722e935aa49 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:46.127562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d4b2-3f6c-4e65-b728-7722e935aa49 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:46.127835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d4b2-3f6c-4e65-b728-7722e935aa49 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:46.128513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125d4b2-3f6c-4e65-b728-7722e935aa49 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:46.129910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125d4b2-3f6c-4e65-b728-7722e935aa49 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:46.131215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 22/43] 10.222.0.25 () {58 vars in 1361 bytes} [Thu Sep 22 21:55:45 2022] PUT /identity/v3/domains/default/users/9a8dd819f6d742ceba5a020c855827f8/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:55:46.132033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8c534-b683-40db-afaa-63f4510abd8a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:46.146261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 22/44] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 22 21:55:45 2022] POST /identity/v3/roles => generated 223 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:55:46.190871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:46.268030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8c534-b683-40db-afaa-63f4510abd8a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:46.309808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3609adc-b584-4035-bed1-94eb2ac73eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:46.310519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609adc-b584-4035-bed1-94eb2ac73eca None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:46.310830 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609adc-b584-4035-bed1-94eb2ac73eca None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:46.311110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3609adc-b584-4035-bed1-94eb2ac73eca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:46.312408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3609adc-b584-4035-bed1-94eb2ac73eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:46.315009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3609adc-b584-4035-bed1-94eb2ac73eca None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:46.317197 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 23/45] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:55:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:46.323132 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d4b2-3f6c-4e65-b728-7722e935aa49 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:46.390431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:46.430449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d4b2-3f6c-4e65-b728-7722e935aa49 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:46.464472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3aa6bd-9dc8-456c-befb-e04351998849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:46.465009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aa6bd-9dc8-456c-befb-e04351998849 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:46.465327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aa6bd-9dc8-456c-befb-e04351998849 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:46.465604 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aa6bd-9dc8-456c-befb-e04351998849 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:46.466264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3aa6bd-9dc8-456c-befb-e04351998849 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:46.467715 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3aa6bd-9dc8-456c-befb-e04351998849 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:46.495909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 23/46] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:55:46 2022] POST /identity/v3/projects => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:55:48.319298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66054-dca5-48ba-b2ed-74c2d2f6f55e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:48.319784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66054-dca5-48ba-b2ed-74c2d2f6f55e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:48.319784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66054-dca5-48ba-b2ed-74c2d2f6f55e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:48.321123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 24/47] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:48.326318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe4cbf-110e-4287-817f-0eff9477d76f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:48.326714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe4cbf-110e-4287-817f-0eff9477d76f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:48.326881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe4cbf-110e-4287-817f-0eff9477d76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:48.346558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9bfe4cbf-110e-4287-817f-0eff9477d76f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:48.348081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9bfe4cbf-110e-4287-817f-0eff9477d76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:48.415538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 24/48] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:48.690901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66054-dca5-48ba-b2ed-74c2d2f6f55e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:48.693898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d49ff-1b8b-4571-adfd-16e4ad176e3a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:48.694248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d49ff-1b8b-4571-adfd-16e4ad176e3a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:48.694582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086d49ff-1b8b-4571-adfd-16e4ad176e3a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:48.696303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 25/49] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:55:48.701267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c40ba-ba4e-4157-824d-d101444a8c8f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:48.701548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c40ba-ba4e-4157-824d-d101444a8c8f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:48.701835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c40ba-ba4e-4157-824d-d101444a8c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:48.726598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e52c40ba-ba4e-4157-824d-d101444a8c8f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:48.727784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e52c40ba-ba4e-4157-824d-d101444a8c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:48.766667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:48.793072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 26/50] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:55:48.793716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66054-dca5-48ba-b2ed-74c2d2f6f55e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:48.802912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:48.827320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df4688-df44-4f1e-9c8c-1eb0c3b46954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:48.827953 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df4688-df44-4f1e-9c8c-1eb0c3b46954 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:48.828239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df4688-df44-4f1e-9c8c-1eb0c3b46954 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:48.829064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72df4688-df44-4f1e-9c8c-1eb0c3b46954 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:48.839333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72df4688-df44-4f1e-9c8c-1eb0c3b46954 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-72df4688-df44-4f1e-9c8c-1eb0c3b46954 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:55:48.845053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:55:48.845962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:55:48.846716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 25/51] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:55:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:55:48.852054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c40ba-ba4e-4157-824d-d101444a8c8f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:48.948485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c40ba-ba4e-4157-824d-d101444a8c8f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:48.982774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49f128-832e-4acc-a517-b691132439f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:48.983388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f128-832e-4acc-a517-b691132439f0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:48.983670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f128-832e-4acc-a517-b691132439f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:48.983946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c49f128-832e-4acc-a517-b691132439f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:48.984602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49f128-832e-4acc-a517-b691132439f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:48.984986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49f128-832e-4acc-a517-b691132439f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:48.986394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c49f128-832e-4acc-a517-b691132439f0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:48.997929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 27/52] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:55:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:49.003660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df4688-df44-4f1e-9c8c-1eb0c3b46954 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:49.059323 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49f128-832e-4acc-a517-b691132439f0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:49.096211 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:49.110986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72df4688-df44-4f1e-9c8c-1eb0c3b46954 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:49.144687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebeb6c-cce6-45d5-b066-a6746c478415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:49.145389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebeb6c-cce6-45d5-b066-a6746c478415 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:49.145665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebeb6c-cce6-45d5-b066-a6746c478415 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:49.145935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebeb6c-cce6-45d5-b066-a6746c478415 None admin] PATH_INFO: `/v3/users/9a8dd819f6d742ceba5a020c855827f8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:49.147047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebeb6c-cce6-45d5-b066-a6746c478415 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8dd819f6d742ceba5a020c855827f8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:49.148521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebeb6c-cce6-45d5-b066-a6746c478415 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:49.150126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 26/53] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:55:49 2022] GET /identity/v3/users/9a8dd819f6d742ceba5a020c855827f8 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:49.155393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebeb6c-cce6-45d5-b066-a6746c478415 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:49.162931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c49f128-832e-4acc-a517-b691132439f0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:49.199852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07599d-221f-4609-8dd5-3c3790c6177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:49.200423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07599d-221f-4609-8dd5-3c3790c6177f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:49.200706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07599d-221f-4609-8dd5-3c3790c6177f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:49.200981 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a07599d-221f-4609-8dd5-3c3790c6177f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:49.202195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07599d-221f-4609-8dd5-3c3790c6177f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:49.204379 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a07599d-221f-4609-8dd5-3c3790c6177f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:49.206274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 28/54] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:55:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:55:49.212041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07599d-221f-4609-8dd5-3c3790c6177f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:49.295110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:49.315830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a07599d-221f-4609-8dd5-3c3790c6177f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:49.318156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:49.350306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa01e08-c678-4546-8026-8904a680aeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:49.350936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa01e08-c678-4546-8026-8904a680aeef None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:49.351176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa01e08-c678-4546-8026-8904a680aeef None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:49.351445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa01e08-c678-4546-8026-8904a680aeef None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:49.352083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa01e08-c678-4546-8026-8904a680aeef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:49.353564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa01e08-c678-4546-8026-8904a680aeef None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:49.359299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebeb6c-cce6-45d5-b066-a6746c478415 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:49.379428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 29/55] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Sep 22 21:55:49 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:55:49.394976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385447d2-8ed9-4659-96e8-5a93fa09dddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:49.395634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385447d2-8ed9-4659-96e8-5a93fa09dddb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:49.395913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385447d2-8ed9-4659-96e8-5a93fa09dddb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:49.396186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385447d2-8ed9-4659-96e8-5a93fa09dddb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:49.397686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385447d2-8ed9-4659-96e8-5a93fa09dddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:49.399216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385447d2-8ed9-4659-96e8-5a93fa09dddb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:49.400876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 27/56] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:55:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:55:49.407698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa01e08-c678-4546-8026-8904a680aeef None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:49.515418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa01e08-c678-4546-8026-8904a680aeef None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:49.551327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6c25d-f5f6-411a-a554-3339ffc90489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:49.551947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6c25d-f5f6-411a-a554-3339ffc90489 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:49.552236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6c25d-f5f6-411a-a554-3339ffc90489 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:49.552513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6c25d-f5f6-411a-a554-3339ffc90489 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:49.553275 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6c25d-f5f6-411a-a554-3339ffc90489 None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.domain.id=default, user.id=9a8dd819f6d742ceba5a020c855827f8) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:49.553585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6c25d-f5f6-411a-a554-3339ffc90489 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:49.555342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6c25d-f5f6-411a-a554-3339ffc90489 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:49.562147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 30/57] 10.222.0.25 () {56 vars in 1322 bytes} [Thu Sep 22 21:55:49 2022] GET /identity/v3/role_assignments?user.id=9a8dd819f6d742ceba5a020c855827f8&scope.domain.id=default&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 530 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:51.546524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112e3f0-efb3-4c14-88e9-b6b7e473cc9d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:51.547057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112e3f0-efb3-4c14-88e9-b6b7e473cc9d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:51.547153 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112e3f0-efb3-4c14-88e9-b6b7e473cc9d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:51.548543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 28/58] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:51.553797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920cf37-f1ac-4f6a-817d-c3b77ee552de None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:51.554207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920cf37-f1ac-4f6a-817d-c3b77ee552de None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:51.554355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920cf37-f1ac-4f6a-817d-c3b77ee552de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:51.572804 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2920cf37-f1ac-4f6a-817d-c3b77ee552de None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:51.574036 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2920cf37-f1ac-4f6a-817d-c3b77ee552de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:51.588212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:51.639660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 31/59] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:51.905912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112e3f0-efb3-4c14-88e9-b6b7e473cc9d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:52.006529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112e3f0-efb3-4c14-88e9-b6b7e473cc9d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:52.008785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:52.041506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68867387-994b-4be8-a1df-697d9bb5695b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:52.042128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68867387-994b-4be8-a1df-697d9bb5695b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:52.042411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68867387-994b-4be8-a1df-697d9bb5695b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:52.042687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68867387-994b-4be8-a1df-697d9bb5695b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:52.043797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68867387-994b-4be8-a1df-697d9bb5695b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:52.045268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68867387-994b-4be8-a1df-697d9bb5695b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:52.046836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 29/60] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:55:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:52.052573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920cf37-f1ac-4f6a-817d-c3b77ee552de None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:52.149189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920cf37-f1ac-4f6a-817d-c3b77ee552de None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:52.180880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf6e6a2-6ddf-4041-9e78-109f5818b3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:52.181463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6e6a2-6ddf-4041-9e78-109f5818b3f0 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:52.181745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6e6a2-6ddf-4041-9e78-109f5818b3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:52.182021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf6e6a2-6ddf-4041-9e78-109f5818b3f0 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:52.182674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6e6a2-6ddf-4041-9e78-109f5818b3f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:52.184114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf6e6a2-6ddf-4041-9e78-109f5818b3f0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:52.210296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 32/61] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:55:52 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:55:55.461039 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deac0bb-83cd-4642-b0ea-d6e3578f4b68 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:55.461909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deac0bb-83cd-4642-b0ea-d6e3578f4b68 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:55.462349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deac0bb-83cd-4642-b0ea-d6e3578f4b68 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:55.463844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 30/62] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:55.469768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651388a-0ef1-401f-b57c-49e51da1fa20 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:55.470544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651388a-0ef1-401f-b57c-49e51da1fa20 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:55.470982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1651388a-0ef1-401f-b57c-49e51da1fa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:55.486799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:55.494918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1651388a-0ef1-401f-b57c-49e51da1fa20 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:55.502466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1651388a-0ef1-401f-b57c-49e51da1fa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:55.585562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 33/63] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:55.627898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdbcbe-8e75-4665-ae42-cced4495061f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:55.628194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdbcbe-8e75-4665-ae42-cced4495061f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:55.628459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdbcbe-8e75-4665-ae42-cced4495061f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:55.630005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 31/64] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:55.635418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b08a33-4145-4470-a71a-82208eeb0873 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:55.635708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b08a33-4145-4470-a71a-82208eeb0873 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:55.635985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b08a33-4145-4470-a71a-82208eeb0873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:55.654585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f3b08a33-4145-4470-a71a-82208eeb0873 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:55.655799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f3b08a33-4145-4470-a71a-82208eeb0873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:55.746556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 34/65] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:55.908242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdbcbe-8e75-4665-ae42-cced4495061f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:55.913703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cadb8-1e06-44b4-adee-dbb7093562f8 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:55.913998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cadb8-1e06-44b4-adee-dbb7093562f8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:55.914277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1cadb8-1e06-44b4-adee-dbb7093562f8 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:55.915847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 35/66] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:55.928363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c818a-75be-496f-8bb5-b29e8430fa3e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:55.928908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c818a-75be-496f-8bb5-b29e8430fa3e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:55.929635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c818a-75be-496f-8bb5-b29e8430fa3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:55.966167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2d5c818a-75be-496f-8bb5-b29e8430fa3e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:55.967679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2d5c818a-75be-496f-8bb5-b29e8430fa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:56.064557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdbcbe-8e75-4665-ae42-cced4495061f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:56.091065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:56.105419 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 36/67] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:55:56.116732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c818a-75be-496f-8bb5-b29e8430fa3e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:56.139763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171da0c-e51a-4103-8a6e-82f4f8cd76c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:56.140425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171da0c-e51a-4103-8a6e-82f4f8cd76c9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.140780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171da0c-e51a-4103-8a6e-82f4f8cd76c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.141057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171da0c-e51a-4103-8a6e-82f4f8cd76c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.165544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171da0c-e51a-4103-8a6e-82f4f8cd76c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b171da0c-e51a-4103-8a6e-82f4f8cd76c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:55:56.170686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:55:56.171590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:55:56.172429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 32/68] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:55:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:55:56.179042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171da0c-e51a-4103-8a6e-82f4f8cd76c9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:56.213611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:56.264887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c818a-75be-496f-8bb5-b29e8430fa3e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:56.300427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171da0c-e51a-4103-8a6e-82f4f8cd76c9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:56.301971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551c4be-f7a8-4991-9aa9-b7a35a0b7704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:56.302599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551c4be-f7a8-4991-9aa9-b7a35a0b7704 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.302889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551c4be-f7a8-4991-9aa9-b7a35a0b7704 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.303163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551c4be-f7a8-4991-9aa9-b7a35a0b7704 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.316463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551c4be-f7a8-4991-9aa9-b7a35a0b7704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f551c4be-f7a8-4991-9aa9-b7a35a0b7704 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:55:56.322848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:55:56.323680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:55:56.324416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 37/69] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:55:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:55:56.325576 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2faaa-bd45-4284-928a-f536f4b6c4fd None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.325863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2faaa-bd45-4284-928a-f536f4b6c4fd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.326142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2faaa-bd45-4284-928a-f536f4b6c4fd None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.327520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 38/70] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:56.330514 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b42d-6250-4ed2-948a-410652353896 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.331478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b42d-6250-4ed2-948a-410652353896 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.332007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9b42d-6250-4ed2-948a-410652353896 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.338128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 39/71] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:55:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:55:56.338128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9b42d-6250-4ed2-948a-410652353896 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:56.346211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2f649-5116-4f26-8f5a-00bfabe50757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:56.346829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2f649-5116-4f26-8f5a-00bfabe50757 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.347110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2f649-5116-4f26-8f5a-00bfabe50757 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.347384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2f649-5116-4f26-8f5a-00bfabe50757 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.348170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2f649-5116-4f26-8f5a-00bfabe50757 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:56.348355 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2f649-5116-4f26-8f5a-00bfabe50757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:56.349769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2f649-5116-4f26-8f5a-00bfabe50757 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:56.358378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 33/72] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:55:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:55:56.364578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3e709-8599-4f0d-98bf-1036d4c77ef0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.364928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3e709-8599-4f0d-98bf-1036d4c77ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.365168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a3e709-8599-4f0d-98bf-1036d4c77ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.383696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-32a3e709-8599-4f0d-98bf-1036d4c77ef0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:56.384903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-32a3e709-8599-4f0d-98bf-1036d4c77ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:56.400056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:56.446173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9b42d-6250-4ed2-948a-410652353896 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:56.454341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 34/73] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:56.455945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c7764-f6f2-4cf7-986f-4f305fd9a4ed None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.456236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c7764-f6f2-4cf7-986f-4f305fd9a4ed None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.456826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c7764-f6f2-4cf7-986f-4f305fd9a4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.477177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7d2c7764-f6f2-4cf7-986f-4f305fd9a4ed None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:55:56.478369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7d2c7764-f6f2-4cf7-986f-4f305fd9a4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:55:56.479770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540aecaa-2e22-47cf-8c89-30461a4c74f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:56.480372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540aecaa-2e22-47cf-8c89-30461a4c74f1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.480645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540aecaa-2e22-47cf-8c89-30461a4c74f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.480914 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540aecaa-2e22-47cf-8c89-30461a4c74f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.481596 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540aecaa-2e22-47cf-8c89-30461a4c74f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:56.481925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540aecaa-2e22-47cf-8c89-30461a4c74f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:56.483305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540aecaa-2e22-47cf-8c89-30461a4c74f1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:56.492396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 40/74] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:55:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:56.492983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540aecaa-2e22-47cf-8c89-30461a4c74f1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:56.502759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:56.555132 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 35/75] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:55:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:55:56.556329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c7764-f6f2-4cf7-986f-4f305fd9a4ed None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:56.608516 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540aecaa-2e22-47cf-8c89-30461a4c74f1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:56.628502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:56.651076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b24ac9-2976-4c79-b49d-f268dbb87536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:56.652125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24ac9-2976-4c79-b49d-f268dbb87536 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.652600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24ac9-2976-4c79-b49d-f268dbb87536 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.653193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b24ac9-2976-4c79-b49d-f268dbb87536 None admin] PATH_INFO: `/v3/users/68d47cfe578c4b96a86d50549ee2c4ed` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.669254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b24ac9-2976-4c79-b49d-f268dbb87536 None admin] RBAC: Authorizing `identity:get_user(user_id=68d47cfe578c4b96a86d50549ee2c4ed)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:56.671124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b24ac9-2976-4c79-b49d-f268dbb87536 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:56.673329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 41/76] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:55:56 2022] GET /identity/v3/users/68d47cfe578c4b96a86d50549ee2c4ed => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:56.674345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b24ac9-2976-4c79-b49d-f268dbb87536 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:56.697032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c7764-f6f2-4cf7-986f-4f305fd9a4ed None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:56.740568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb116f6-e063-4117-ae62-dd86a2634bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:56.741222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb116f6-e063-4117-ae62-dd86a2634bcb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.741511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb116f6-e063-4117-ae62-dd86a2634bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.741950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb116f6-e063-4117-ae62-dd86a2634bcb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.752530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb116f6-e063-4117-ae62-dd86a2634bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cdb116f6-e063-4117-ae62-dd86a2634bcb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:55:56.757368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:55:56.758227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:55:56.758977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 36/77] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:55:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:55:56.763898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb116f6-e063-4117-ae62-dd86a2634bcb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:56.780908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:56.783006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b24ac9-2976-4c79-b49d-f268dbb87536 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:56.819998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9ffae-177c-4752-8709-694a45ffd059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:56.820636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9ffae-177c-4752-8709-694a45ffd059 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.820921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9ffae-177c-4752-8709-694a45ffd059 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.821228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a9ffae-177c-4752-8709-694a45ffd059 None admin] PATH_INFO: `/v3/users/9a8dd819f6d742ceba5a020c855827f8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.822806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9ffae-177c-4752-8709-694a45ffd059 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8dd819f6d742ceba5a020c855827f8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:56.824250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a9ffae-177c-4752-8709-694a45ffd059 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:56.825874 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 42/78] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:55:56 2022] GET /identity/v3/users/9a8dd819f6d742ceba5a020c855827f8 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:56.829306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9ffae-177c-4752-8709-694a45ffd059 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:56.873536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:56.874404 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb116f6-e063-4117-ae62-dd86a2634bcb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:56.909761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540f704-089d-4c8d-9a3f-f116dc23f76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:56.911423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540f704-089d-4c8d-9a3f-f116dc23f76b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.911712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540f704-089d-4c8d-9a3f-f116dc23f76b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.911989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540f704-089d-4c8d-9a3f-f116dc23f76b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.927231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a9ffae-177c-4752-8709-694a45ffd059 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:56.961011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cebd3-e618-4df1-a773-a83b1de31b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:56.961690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cebd3-e618-4df1-a773-a83b1de31b68 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:56.961975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cebd3-e618-4df1-a773-a83b1de31b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:56.962253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2cebd3-e618-4df1-a773-a83b1de31b68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:56.972529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2cebd3-e618-4df1-a773-a83b1de31b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-4d2cebd3-e618-4df1-a773-a83b1de31b68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:55:56.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:55:56.978818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:55:56.979711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:55:56.980331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540f704-089d-4c8d-9a3f-f116dc23f76b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:56.980399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 43/79] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:55:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:55:56.980460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540f704-089d-4c8d-9a3f-f116dc23f76b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:56.981620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cebd3-e618-4df1-a773-a83b1de31b68 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:56.995259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:57.080491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 37/80] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:55:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:55:57.081338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2cebd3-e618-4df1-a773-a83b1de31b68 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:57.081445 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540f704-089d-4c8d-9a3f-f116dc23f76b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:57.083810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:57.115765 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebb447-5721-4c2f-a55c-fa692a2284a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:57.116370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebb447-5721-4c2f-a55c-fa692a2284a0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:57.116654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebb447-5721-4c2f-a55c-fa692a2284a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:57.116929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebb447-5721-4c2f-a55c-fa692a2284a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:57.117627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebb447-5721-4c2f-a55c-fa692a2284a0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:57.117933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebb447-5721-4c2f-a55c-fa692a2284a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:57.119305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebb447-5721-4c2f-a55c-fa692a2284a0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:57.127371 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 44/81] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:55:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:57.129965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebb447-5721-4c2f-a55c-fa692a2284a0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:57.182101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540f704-089d-4c8d-9a3f-f116dc23f76b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:57.195494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:57.215867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc0c22-966f-481d-b158-54f1655c043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:57.216482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc0c22-966f-481d-b158-54f1655c043e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:57.216757 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc0c22-966f-481d-b158-54f1655c043e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:57.217034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc0c22-966f-481d-b158-54f1655c043e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:57.228720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebb447-5721-4c2f-a55c-fa692a2284a0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:57.261724 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c5979-7ed4-476f-92ba-28578610b485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:57.262351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c5979-7ed4-476f-92ba-28578610b485 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:57.262631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c5979-7ed4-476f-92ba-28578610b485 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:57.262905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c5979-7ed4-476f-92ba-28578610b485 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:57.264010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c5979-7ed4-476f-92ba-28578610b485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:57.266015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c5979-7ed4-476f-92ba-28578610b485 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:57.267569 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 45/82] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:55:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:57.268321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c5979-7ed4-476f-92ba-28578610b485 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:57.284966 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc0c22-966f-481d-b158-54f1655c043e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:57.286442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfc0c22-966f-481d-b158-54f1655c043e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:57.303184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:57.368610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c5979-7ed4-476f-92ba-28578610b485 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:57.383753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 38/83] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:55:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:55:57.384618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc0c22-966f-481d-b158-54f1655c043e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:57.399384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:57.406650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016efbb-49f1-4c31-aded-a798a429e132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:57.407254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016efbb-49f1-4c31-aded-a798a429e132 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:57.407531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016efbb-49f1-4c31-aded-a798a429e132 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:57.407798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016efbb-49f1-4c31-aded-a798a429e132 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:57.408447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016efbb-49f1-4c31-aded-a798a429e132 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:57.408748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016efbb-49f1-4c31-aded-a798a429e132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:57.410148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016efbb-49f1-4c31-aded-a798a429e132 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:57.418399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 46/84] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:55:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:57.418399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016efbb-49f1-4c31-aded-a798a429e132 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:57.481730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc0c22-966f-481d-b158-54f1655c043e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:57.512152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:57.516251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edfd8c-d3ad-4d02-bbe6-ad6f7100d6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:57.516884 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edfd8c-d3ad-4d02-bbe6-ad6f7100d6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:57.517198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edfd8c-d3ad-4d02-bbe6-ad6f7100d6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:57.517477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17edfd8c-d3ad-4d02-bbe6-ad6f7100d6a7 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:57.518610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edfd8c-d3ad-4d02-bbe6-ad6f7100d6a7 None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:57.519755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016efbb-49f1-4c31-aded-a798a429e132 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:57.520045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17edfd8c-d3ad-4d02-bbe6-ad6f7100d6a7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:57.522647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 39/85] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:55:57 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:57.522647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edfd8c-d3ad-4d02-bbe6-ad6f7100d6a7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:57.553916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfc5de-2153-4cdf-9fff-d5942e1f7b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:57.554541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfc5de-2153-4cdf-9fff-d5942e1f7b8c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:57.554822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfc5de-2153-4cdf-9fff-d5942e1f7b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:57.555100 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dfc5de-2153-4cdf-9fff-d5942e1f7b8c None admin] PATH_INFO: `/v3/users/68d47cfe578c4b96a86d50549ee2c4ed` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:57.556215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfc5de-2153-4cdf-9fff-d5942e1f7b8c None admin] RBAC: Authorizing `identity:get_user(user_id=68d47cfe578c4b96a86d50549ee2c4ed)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:57.557687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dfc5de-2153-4cdf-9fff-d5942e1f7b8c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:57.559262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 47/86] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:55:57 2022] GET /identity/v3/users/68d47cfe578c4b96a86d50549ee2c4ed => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:57.560027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfc5de-2153-4cdf-9fff-d5942e1f7b8c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:57.612300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:57.630991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17edfd8c-d3ad-4d02-bbe6-ad6f7100d6a7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:57.658305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dfc5de-2153-4cdf-9fff-d5942e1f7b8c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:57.664713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f438b5-3015-406b-8a39-0b11ac7c7bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:57.665323 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f438b5-3015-406b-8a39-0b11ac7c7bb4 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:57.665597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f438b5-3015-406b-8a39-0b11ac7c7bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:57.665865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f438b5-3015-406b-8a39-0b11ac7c7bb4 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:57.666511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f438b5-3015-406b-8a39-0b11ac7c7bb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:57.667934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f438b5-3015-406b-8a39-0b11ac7c7bb4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:57.692198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9cb45-b3db-442e-bb7d-a870c80cc527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:57.692837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9cb45-b3db-442e-bb7d-a870c80cc527 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:57.693149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9cb45-b3db-442e-bb7d-a870c80cc527 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:57.693429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9cb45-b3db-442e-bb7d-a870c80cc527 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:57.694485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 40/87] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:55:57 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:55:57.694597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9cb45-b3db-442e-bb7d-a870c80cc527 None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:57.695285 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f438b5-3015-406b-8a39-0b11ac7c7bb4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:57.696134 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9cb45-b3db-442e-bb7d-a870c80cc527 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:57.697828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 48/88] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:55:57 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:57.697828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9cb45-b3db-442e-bb7d-a870c80cc527 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:57.736384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:57.809235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9cb45-b3db-442e-bb7d-a870c80cc527 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:57.821074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f438b5-3015-406b-8a39-0b11ac7c7bb4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:57.845828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc426d-e384-4ef7-a9e2-a75525f6f2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:57.846451 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc426d-e384-4ef7-a9e2-a75525f6f2aa None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:57.846736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc426d-e384-4ef7-a9e2-a75525f6f2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:57.847015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcc426d-e384-4ef7-a9e2-a75525f6f2aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:57.847866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc426d-e384-4ef7-a9e2-a75525f6f2aa None admin] RBAC: Adding query filter params (role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, scope.project.id=8087b41e58fb4aca97cbfced2c011450, user.id=68d47cfe578c4b96a86d50549ee2c4ed) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:57.848099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc426d-e384-4ef7-a9e2-a75525f6f2aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:57.849621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcc426d-e384-4ef7-a9e2-a75525f6f2aa None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:57.855398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 49/89] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:55:57 2022] GET /identity/v3/role_assignments?user.id=68d47cfe578c4b96a86d50549ee2c4ed&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:57.856248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc426d-e384-4ef7-a9e2-a75525f6f2aa None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:57.872488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27341745-5a19-4859-aef8-dd09cbc760bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:57.873539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27341745-5a19-4859-aef8-dd09cbc760bb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:57.873822 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27341745-5a19-4859-aef8-dd09cbc760bb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:57.874096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27341745-5a19-4859-aef8-dd09cbc760bb None admin] PATH_INFO: `/v3/users/68d47cfe578c4b96a86d50549ee2c4ed` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:57.877187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27341745-5a19-4859-aef8-dd09cbc760bb None admin] RBAC: Authorizing `identity:get_user(user_id=68d47cfe578c4b96a86d50549ee2c4ed)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:57.879248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27341745-5a19-4859-aef8-dd09cbc760bb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:57.881395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 41/90] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:55:57 2022] GET /identity/v3/users/68d47cfe578c4b96a86d50549ee2c4ed => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:57.882204 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27341745-5a19-4859-aef8-dd09cbc760bb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:57.892808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:57.990712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:57.997542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcc426d-e384-4ef7-a9e2-a75525f6f2aa None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:58.032996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27341745-5a19-4859-aef8-dd09cbc760bb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:58.042964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a8753-96e2-4c78-8ded-2d0000e48070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:58.043835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a8753-96e2-4c78-8ded-2d0000e48070 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:58.044255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a8753-96e2-4c78-8ded-2d0000e48070 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:58.045048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1a8753-96e2-4c78-8ded-2d0000e48070 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:58.088495 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57b6bd-d93e-49d9-9444-43619f950a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:58.089129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57b6bd-d93e-49d9-9444-43619f950a39 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:58.089429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57b6bd-d93e-49d9-9444-43619f950a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:58.090066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57b6bd-d93e-49d9-9444-43619f950a39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:58.090821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57b6bd-d93e-49d9-9444-43619f950a39 None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=8087b41e58fb4aca97cbfced2c011450, user.id=9a8dd819f6d742ceba5a020c855827f8) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:58.091134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57b6bd-d93e-49d9-9444-43619f950a39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:58.092592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57b6bd-d93e-49d9-9444-43619f950a39 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:58.107170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 42/91] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:55:57 2022] GET /identity/v3/role_assignments?user.id=9a8dd819f6d742ceba5a020c855827f8&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:58.108109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57b6bd-d93e-49d9-9444-43619f950a39 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:58.133531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a8753-96e2-4c78-8ded-2d0000e48070 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:58.135462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1a8753-96e2-4c78-8ded-2d0000e48070 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:58.224511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:58.267107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57b6bd-d93e-49d9-9444-43619f950a39 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:58.286165 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:58.294492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 50/92] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:55:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:55:58.306638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294d592-bece-45aa-ada1-2c7e18ed96e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:58.307329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294d592-bece-45aa-ada1-2c7e18ed96e0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:58.307612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294d592-bece-45aa-ada1-2c7e18ed96e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:58.307886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294d592-bece-45aa-ada1-2c7e18ed96e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:58.310550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a8753-96e2-4c78-8ded-2d0000e48070 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:58.382922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294d592-bece-45aa-ada1-2c7e18ed96e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:58.384427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d294d592-bece-45aa-ada1-2c7e18ed96e0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:58.461455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:58.499633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1a8753-96e2-4c78-8ded-2d0000e48070 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:58.521476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 43/93] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:55:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:55:58.527238 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294d592-bece-45aa-ada1-2c7e18ed96e0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:58.572296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17d50c-fda2-4b20-8cc2-becfd409e0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:58.572993 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17d50c-fda2-4b20-8cc2-becfd409e0d2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:58.573304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17d50c-fda2-4b20-8cc2-becfd409e0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:58.573582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17d50c-fda2-4b20-8cc2-becfd409e0d2 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:58.575940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17d50c-fda2-4b20-8cc2-becfd409e0d2 None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:58.577432 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17d50c-fda2-4b20-8cc2-becfd409e0d2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:58.584521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 51/94] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:55:58 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:55:58.585444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17d50c-fda2-4b20-8cc2-becfd409e0d2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:58.657423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294d592-bece-45aa-ada1-2c7e18ed96e0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:58.694231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686d108-14c8-494f-ab34-f61c794e5762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:58.694870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686d108-14c8-494f-ab34-f61c794e5762 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:58.695151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686d108-14c8-494f-ab34-f61c794e5762 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:58.695488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f686d108-14c8-494f-ab34-f61c794e5762 None admin] PATH_INFO: `/v3/projects/8d8a80f2366945e4836c5b2979ae10fc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:58.696679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686d108-14c8-494f-ab34-f61c794e5762 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8a80f2366945e4836c5b2979ae10fc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:58.698158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f686d108-14c8-494f-ab34-f61c794e5762 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:58.699860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 44/95] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:55:58 2022] GET /identity/v3/projects/8d8a80f2366945e4836c5b2979ae10fc => generated 312 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:55:58.707129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686d108-14c8-494f-ab34-f61c794e5762 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:55:58.739578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:58.765619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:55:58.845287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f686d108-14c8-494f-ab34-f61c794e5762 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:58.846972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17d50c-fda2-4b20-8cc2-becfd409e0d2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:55:58.890694 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62e6c20-a881-4981-9f5f-7468b49fea37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:58.891315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e6c20-a881-4981-9f5f-7468b49fea37 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:58.891602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e6c20-a881-4981-9f5f-7468b49fea37 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:58.891884 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62e6c20-a881-4981-9f5f-7468b49fea37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:58.892632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e6c20-a881-4981-9f5f-7468b49fea37 None admin] RBAC: Adding query filter params (role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, scope.project.id=8d8a80f2366945e4836c5b2979ae10fc, user.id=68d47cfe578c4b96a86d50549ee2c4ed) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:58.893121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e6c20-a881-4981-9f5f-7468b49fea37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:58.894753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62e6c20-a881-4981-9f5f-7468b49fea37 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:58.900895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 45/96] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:55:58 2022] GET /identity/v3/role_assignments?user.id=68d47cfe578c4b96a86d50549ee2c4ed&scope.project.id=8d8a80f2366945e4836c5b2979ae10fc&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:55:58.940962 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bc14e3-b047-4663-9a7c-d68942c84d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:55:58.941628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc14e3-b047-4663-9a7c-d68942c84d1d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:55:58.941910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc14e3-b047-4663-9a7c-d68942c84d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:55:58.942180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bc14e3-b047-4663-9a7c-d68942c84d1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:55:58.942920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc14e3-b047-4663-9a7c-d68942c84d1d None admin] RBAC: Adding query filter params (role.id=dea086603b04451680a14ae806205d79, scope.project.id=8087b41e58fb4aca97cbfced2c011450, user.id=68d47cfe578c4b96a86d50549ee2c4ed) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:55:58.943227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc14e3-b047-4663-9a7c-d68942c84d1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:55:58.944732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bc14e3-b047-4663-9a7c-d68942c84d1d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:55:58.958414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 52/97] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:55:58 2022] GET /identity/v3/role_assignments?user.id=68d47cfe578c4b96a86d50549ee2c4ed&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=dea086603b04451680a14ae806205d79 => generated 259 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:00.970539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18d287-d955-4067-8802-7616071ecb63 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:00.971617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18d287-d955-4067-8802-7616071ecb63 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:00.972143 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18d287-d955-4067-8802-7616071ecb63 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:00.973726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 46/98] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:00.979415 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc3ba-e070-495b-b818-8a442546b247 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:00.979834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc3ba-e070-495b-b818-8a442546b247 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:00.980033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320bc3ba-e070-495b-b818-8a442546b247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:01.001461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6161b38-34c3-4475-bbbb-17b841b9188a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:01.002073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6161b38-34c3-4475-bbbb-17b841b9188a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:01.002550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6161b38-34c3-4475-bbbb-17b841b9188a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:01.004112 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 47/99] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:01.005327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-320bc3ba-e070-495b-b818-8a442546b247 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:01.006259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-320bc3ba-e070-495b-b818-8a442546b247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:01.010089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b71b-a28e-436e-a4ca-b347fb4dd0c1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:01.010610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b71b-a28e-436e-a4ca-b347fb4dd0c1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:01.011104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b71b-a28e-436e-a4ca-b347fb4dd0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:01.032033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9e65b71b-a28e-436e-a4ca-b347fb4dd0c1 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:01.033726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9e65b71b-a28e-436e-a4ca-b347fb4dd0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:01.116872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 53/100] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:01.139645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 48/101] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:01.198024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877adba1-ae97-467e-902a-703aa5cfdeca None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:01.198637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877adba1-ae97-467e-902a-703aa5cfdeca None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:01.199124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877adba1-ae97-467e-902a-703aa5cfdeca None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:01.200667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 54/102] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:01.206899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3c7ec-3d95-4f3e-8807-6ab18cc9940d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:01.207192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3c7ec-3d95-4f3e-8807-6ab18cc9940d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:01.207471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3c7ec-3d95-4f3e-8807-6ab18cc9940d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:01.225973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e7a3c7ec-3d95-4f3e-8807-6ab18cc9940d None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:01.227082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e7a3c7ec-3d95-4f3e-8807-6ab18cc9940d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:01.254012 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:01.305303 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 49/103] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:01.498411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877adba1-ae97-467e-902a-703aa5cfdeca None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:01.581521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:01.581697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cefc55-b31e-47a7-9cfb-b35f2a2d6e97 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:01.581870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cefc55-b31e-47a7-9cfb-b35f2a2d6e97 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:01.582381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cefc55-b31e-47a7-9cfb-b35f2a2d6e97 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:01.583967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 50/104] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:01.590196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a95dc-1687-4fa4-a7ba-3d79f07d4f75 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:01.590769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a95dc-1687-4fa4-a7ba-3d79f07d4f75 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:01.591245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a95dc-1687-4fa4-a7ba-3d79f07d4f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:01.625161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0a9a95dc-1687-4fa4-a7ba-3d79f07d4f75 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:01.626724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0a9a95dc-1687-4fa4-a7ba-3d79f07d4f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:01.659677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877adba1-ae97-467e-902a-703aa5cfdeca None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:01.700198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 51/105] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:01.705048 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a95dc-1687-4fa4-a7ba-3d79f07d4f75 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:01.745131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763960b-3f19-45ee-b143-76cc8a2f3af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:01.745131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763960b-3f19-45ee-b143-76cc8a2f3af5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:01.745131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763960b-3f19-45ee-b143-76cc8a2f3af5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:01.745131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0763960b-3f19-45ee-b143-76cc8a2f3af5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:01.773735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0763960b-3f19-45ee-b143-76cc8a2f3af5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0763960b-3f19-45ee-b143-76cc8a2f3af5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:01.778702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:01.779541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:01.780795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 55/106] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:01.781472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763960b-3f19-45ee-b143-76cc8a2f3af5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:01.825532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:01.832777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a95dc-1687-4fa4-a7ba-3d79f07d4f75 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:01.890388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241dd54d-d4e3-4bc6-a015-25ffd1e77065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:01.891248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dd54d-d4e3-4bc6-a015-25ffd1e77065 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:01.891537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dd54d-d4e3-4bc6-a015-25ffd1e77065 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:01.891817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241dd54d-d4e3-4bc6-a015-25ffd1e77065 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:01.911445 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241dd54d-d4e3-4bc6-a015-25ffd1e77065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-241dd54d-d4e3-4bc6-a015-25ffd1e77065 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:01.916362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:01.917205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 52/107] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:56:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:01.918377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241dd54d-d4e3-4bc6-a015-25ffd1e77065 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:01.998369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0763960b-3f19-45ee-b143-76cc8a2f3af5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:02.035230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:02.076138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1714493-b9e0-412b-bf54-66e33d392109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:02.077163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1714493-b9e0-412b-bf54-66e33d392109 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.077812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1714493-b9e0-412b-bf54-66e33d392109 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.078287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1714493-b9e0-412b-bf54-66e33d392109 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.089194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1714493-b9e0-412b-bf54-66e33d392109 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a1714493-b9e0-412b-bf54-66e33d392109 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:02.095184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:02.096060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:02.096822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:02.099430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 56/108] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:02.100519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1714493-b9e0-412b-bf54-66e33d392109 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:02.100809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241dd54d-d4e3-4bc6-a015-25ffd1e77065 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:02.146836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5195a39-8ca4-4e9a-9026-e892ea8bf162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:02.147467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5195a39-8ca4-4e9a-9026-e892ea8bf162 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.147748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5195a39-8ca4-4e9a-9026-e892ea8bf162 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.148022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5195a39-8ca4-4e9a-9026-e892ea8bf162 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.148706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5195a39-8ca4-4e9a-9026-e892ea8bf162 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:02.149014 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5195a39-8ca4-4e9a-9026-e892ea8bf162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:02.151087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5195a39-8ca4-4e9a-9026-e892ea8bf162 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:02.162297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 53/109] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:02.162973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5195a39-8ca4-4e9a-9026-e892ea8bf162 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:02.187815 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:02.216113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1714493-b9e0-412b-bf54-66e33d392109 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:02.258021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a402b-a5d5-458c-b4d1-cc3514e67ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:02.258636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a402b-a5d5-458c-b4d1-cc3514e67ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.258915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a402b-a5d5-458c-b4d1-cc3514e67ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.259200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80a402b-a5d5-458c-b4d1-cc3514e67ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.259847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a402b-a5d5-458c-b4d1-cc3514e67ad6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:02.260156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a402b-a5d5-458c-b4d1-cc3514e67ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:02.261645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80a402b-a5d5-458c-b4d1-cc3514e67ad6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:02.270538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 57/110] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:02.270538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a402b-a5d5-458c-b4d1-cc3514e67ad6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:02.302256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5195a39-8ca4-4e9a-9026-e892ea8bf162 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:02.315113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:02.346070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49987c84-933f-4343-a9e9-20a31bc2978c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:02.346702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49987c84-933f-4343-a9e9-20a31bc2978c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.347000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49987c84-933f-4343-a9e9-20a31bc2978c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.347255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49987c84-933f-4343-a9e9-20a31bc2978c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.348359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49987c84-933f-4343-a9e9-20a31bc2978c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:02.349845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49987c84-933f-4343-a9e9-20a31bc2978c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:02.351571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 54/111] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:02.356610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49987c84-933f-4343-a9e9-20a31bc2978c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:02.379935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80a402b-a5d5-458c-b4d1-cc3514e67ad6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:02.413617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd900fd-4501-4b7d-b991-1fc9138832b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:02.414231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd900fd-4501-4b7d-b991-1fc9138832b2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.414513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd900fd-4501-4b7d-b991-1fc9138832b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.414790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd900fd-4501-4b7d-b991-1fc9138832b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.415449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd900fd-4501-4b7d-b991-1fc9138832b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:02.415746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd900fd-4501-4b7d-b991-1fc9138832b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:02.417129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd900fd-4501-4b7d-b991-1fc9138832b2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:02.426693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 58/112] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:02.428192 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33e65b-42e1-4901-8287-75698c834f89 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.428483 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33e65b-42e1-4901-8287-75698c834f89 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.428760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc33e65b-42e1-4901-8287-75698c834f89 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.430571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 59/113] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:02.434615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33e65b-42e1-4901-8287-75698c834f89 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:02.442936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:02.460935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49987c84-933f-4343-a9e9-20a31bc2978c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:02.494013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61638aea-1bc1-47e7-9c23-03da199dd397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:02.494641 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61638aea-1bc1-47e7-9c23-03da199dd397 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.494919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61638aea-1bc1-47e7-9c23-03da199dd397 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.495193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61638aea-1bc1-47e7-9c23-03da199dd397 None admin] PATH_INFO: `/v3/users/9c1646303f474283bd07c9c93a1887d8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.511240 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61638aea-1bc1-47e7-9c23-03da199dd397 None admin] RBAC: Authorizing `identity:get_user(user_id=9c1646303f474283bd07c9c93a1887d8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:02.512683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61638aea-1bc1-47e7-9c23-03da199dd397 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:02.514323 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 55/114] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:02 2022] GET /identity/v3/users/9c1646303f474283bd07c9c93a1887d8 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:02.517868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61638aea-1bc1-47e7-9c23-03da199dd397 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:02.540324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc33e65b-42e1-4901-8287-75698c834f89 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:02.541858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:02.573544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a85e1f1-ce5c-4ea8-a7d3-1153df750ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:02.574091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85e1f1-ce5c-4ea8-a7d3-1153df750ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.574369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85e1f1-ce5c-4ea8-a7d3-1153df750ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.574639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a85e1f1-ce5c-4ea8-a7d3-1153df750ea3 None admin] PATH_INFO: `/v3/users/9a8dd819f6d742ceba5a020c855827f8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.575754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85e1f1-ce5c-4ea8-a7d3-1153df750ea3 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8dd819f6d742ceba5a020c855827f8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:02.577213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a85e1f1-ce5c-4ea8-a7d3-1153df750ea3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:02.579327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 60/115] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:02 2022] GET /identity/v3/users/9a8dd819f6d742ceba5a020c855827f8 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:02.581574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab597003-aefb-48ff-8ac8-f46f2f656f4d None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.581868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab597003-aefb-48ff-8ac8-f46f2f656f4d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.582138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab597003-aefb-48ff-8ac8-f46f2f656f4d None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.583375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 61/116] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:02.584524 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab597003-aefb-48ff-8ac8-f46f2f656f4d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:02.621217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61638aea-1bc1-47e7-9c23-03da199dd397 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:02.656853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4a10d1-4ee3-4841-9d65-db7140cf30ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:02.657501 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a10d1-4ee3-4841-9d65-db7140cf30ec None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.657783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a10d1-4ee3-4841-9d65-db7140cf30ec None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.658055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4a10d1-4ee3-4841-9d65-db7140cf30ec None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.658702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a10d1-4ee3-4841-9d65-db7140cf30ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:02.660127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4a10d1-4ee3-4841-9d65-db7140cf30ec None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:02.675627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:02.683390 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab597003-aefb-48ff-8ac8-f46f2f656f4d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:02.694162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 56/117] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:56:02 2022] POST /identity/v3/projects => generated 302 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:56:02.696465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bc92d-d02a-4aa1-affa-3b51a50f1bc3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.696991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bc92d-d02a-4aa1-affa-3b51a50f1bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.697518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bc92d-d02a-4aa1-affa-3b51a50f1bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.742503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0d6bc92d-d02a-4aa1-affa-3b51a50f1bc3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:02.744063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0d6bc92d-d02a-4aa1-affa-3b51a50f1bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:02.750641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df8a46-d8b3-4ff0-8125-0e18d617c78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:02.751517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df8a46-d8b3-4ff0-8125-0e18d617c78a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.751873 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df8a46-d8b3-4ff0-8125-0e18d617c78a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.752157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df8a46-d8b3-4ff0-8125-0e18d617c78a None admin] PATH_INFO: `/v3/users/68d47cfe578c4b96a86d50549ee2c4ed` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:02.753559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df8a46-d8b3-4ff0-8125-0e18d617c78a None admin] RBAC: Authorizing `identity:get_user(user_id=68d47cfe578c4b96a86d50549ee2c4ed)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:02.755097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df8a46-d8b3-4ff0-8125-0e18d617c78a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:02.761446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 62/118] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:02 2022] GET /identity/v3/users/68d47cfe578c4b96a86d50549ee2c4ed => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:02.763590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df8a46-d8b3-4ff0-8125-0e18d617c78a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:02.829529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:02.869161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 57/119] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:02.869161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bc92d-d02a-4aa1-affa-3b51a50f1bc3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:02.916689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df8a46-d8b3-4ff0-8125-0e18d617c78a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:02.958833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a3f04-bc8f-4763-a1ba-3e670ba5a6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:02.960000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a3f04-bc8f-4763-a1ba-3e670ba5a6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:02.960479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a3f04-bc8f-4763-a1ba-3e670ba5a6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:02.960920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a3f04-bc8f-4763-a1ba-3e670ba5a6b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:03.005921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bc92d-d02a-4aa1-affa-3b51a50f1bc3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:03.047929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:03.058011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a3f04-bc8f-4763-a1ba-3e670ba5a6b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:03.061399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a3f04-bc8f-4763-a1ba-3e670ba5a6b5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:03.072626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0efbee-d975-44c6-b4dc-9bb29dea364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:03.073335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0efbee-d975-44c6-b4dc-9bb29dea364b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:03.073625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0efbee-d975-44c6-b4dc-9bb29dea364b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:03.073903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0efbee-d975-44c6-b4dc-9bb29dea364b None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:03.079212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0efbee-d975-44c6-b4dc-9bb29dea364b None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:03.080745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0efbee-d975-44c6-b4dc-9bb29dea364b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:03.082530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 58/120] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:02 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:03.086188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6af9a-cf8d-4e17-9d13-30e790cb9508 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:03.086487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6af9a-cf8d-4e17-9d13-30e790cb9508 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:03.086762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6af9a-cf8d-4e17-9d13-30e790cb9508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:03.126657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:03.130596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5bf6af9a-cf8d-4e17-9d13-30e790cb9508 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:03.133168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5bf6af9a-cf8d-4e17-9d13-30e790cb9508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:03.184780 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 63/121] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:03.185763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a3f04-bc8f-4763-a1ba-3e670ba5a6b5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:03.230806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 59/122] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:03.238015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6af9a-cf8d-4e17-9d13-30e790cb9508 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:03.290701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a3f04-bc8f-4763-a1ba-3e670ba5a6b5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:03.326919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dd916-f7f7-4ca5-92a8-a74eb3611c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:03.328591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dd916-f7f7-4ca5-92a8-a74eb3611c86 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:03.328591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dd916-f7f7-4ca5-92a8-a74eb3611c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:03.328837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dd916-f7f7-4ca5-92a8-a74eb3611c86 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:03.330062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97dd916-f7f7-4ca5-92a8-a74eb3611c86 None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:03.331661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97dd916-f7f7-4ca5-92a8-a74eb3611c86 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:03.338285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 64/123] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:03 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:03.339034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dd916-f7f7-4ca5-92a8-a74eb3611c86 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:03.348542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:03.376809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6af9a-cf8d-4e17-9d13-30e790cb9508 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:03.405623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:03.433617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f97d2-7beb-49d3-957e-daaf220d6221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:03.434295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f97d2-7beb-49d3-957e-daaf220d6221 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:03.435193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f97d2-7beb-49d3-957e-daaf220d6221 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:03.435193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f97d2-7beb-49d3-957e-daaf220d6221 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450/users/9a8dd819f6d742ceba5a020c855827f8/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:03.436741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f97d2-7beb-49d3-957e-daaf220d6221 None admin] RBAC: Authorizing `identity:create_grant(project_id=8087b41e58fb4aca97cbfced2c011450, user_id=9a8dd819f6d742ceba5a020c855827f8, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:03.438808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f97d2-7beb-49d3-957e-daaf220d6221 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:03.452233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 60/124] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:56:03 2022] PUT /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450/users/9a8dd819f6d742ceba5a020c855827f8/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:03.457357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f97d2-7beb-49d3-957e-daaf220d6221 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:03.487806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dd916-f7f7-4ca5-92a8-a74eb3611c86 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:03.548833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89a2a-d2ec-4fd8-857a-f985ed5601ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:03.549632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89a2a-d2ec-4fd8-857a-f985ed5601ee None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:03.549979 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89a2a-d2ec-4fd8-857a-f985ed5601ee None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:03.550826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d89a2a-d2ec-4fd8-857a-f985ed5601ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:03.567010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d89a2a-d2ec-4fd8-857a-f985ed5601ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-98d89a2a-d2ec-4fd8-857a-f985ed5601ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:03.575359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:03.576293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:03.577087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 65/125] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:03.582579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89a2a-d2ec-4fd8-857a-f985ed5601ee None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:03.584424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f97d2-7beb-49d3-957e-daaf220d6221 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:03.628140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da39bd-5486-4a00-8fab-0918be69c34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:03.629078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da39bd-5486-4a00-8fab-0918be69c34e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:03.629533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da39bd-5486-4a00-8fab-0918be69c34e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:03.629952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da39bd-5486-4a00-8fab-0918be69c34e None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:03.631661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da39bd-5486-4a00-8fab-0918be69c34e None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:03.633376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da39bd-5486-4a00-8fab-0918be69c34e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:03.635598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 61/126] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:03 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:03.636615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da39bd-5486-4a00-8fab-0918be69c34e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:03.665698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:03.696873 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:03.714860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d89a2a-d2ec-4fd8-857a-f985ed5601ee None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:03.756437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefb180-ee54-40d2-a859-f0b421bf6e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:03.757502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefb180-ee54-40d2-a859-f0b421bf6e19 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:03.757996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefb180-ee54-40d2-a859-f0b421bf6e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:03.758477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fefb180-ee54-40d2-a859-f0b421bf6e19 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450/users/68d47cfe578c4b96a86d50549ee2c4ed/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:03.758581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da39bd-5486-4a00-8fab-0918be69c34e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:03.761080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fefb180-ee54-40d2-a859-f0b421bf6e19 None admin] RBAC: Authorizing `identity:create_grant(project_id=8087b41e58fb4aca97cbfced2c011450, user_id=68d47cfe578c4b96a86d50549ee2c4ed, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:03.762900 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fefb180-ee54-40d2-a859-f0b421bf6e19 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:03.782521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 66/127] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:56:03 2022] PUT /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450/users/68d47cfe578c4b96a86d50549ee2c4ed/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:03.783833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefb180-ee54-40d2-a859-f0b421bf6e19 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:03.817991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586b6f8-d4fc-4fe1-80f9-e52a9694c4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:03.818653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586b6f8-d4fc-4fe1-80f9-e52a9694c4ec None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:03.818976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586b6f8-d4fc-4fe1-80f9-e52a9694c4ec None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:03.819262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9586b6f8-d4fc-4fe1-80f9-e52a9694c4ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:03.849595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9586b6f8-d4fc-4fe1-80f9-e52a9694c4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9586b6f8-d4fc-4fe1-80f9-e52a9694c4ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:03.858011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:03.859478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:03.860312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:03.862696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 62/128] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:03.864026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586b6f8-d4fc-4fe1-80f9-e52a9694c4ec None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:03.974681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fefb180-ee54-40d2-a859-f0b421bf6e19 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:04.020675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:04.041441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd78b1-02eb-44f2-b2eb-dc566eb0bcce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:04.042415 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd78b1-02eb-44f2-b2eb-dc566eb0bcce None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:04.042888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd78b1-02eb-44f2-b2eb-dc566eb0bcce None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:04.043374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bd78b1-02eb-44f2-b2eb-dc566eb0bcce None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:04.044227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd78b1-02eb-44f2-b2eb-dc566eb0bcce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:04.044718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd78b1-02eb-44f2-b2eb-dc566eb0bcce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:04.048113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bd78b1-02eb-44f2-b2eb-dc566eb0bcce None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:04.053855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9586b6f8-d4fc-4fe1-80f9-e52a9694c4ec None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:04.064569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:04.071511 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 67/129] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:04.073190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd78b1-02eb-44f2-b2eb-dc566eb0bcce None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:04.103063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ea09b-b1c5-4440-98e1-6114fe4837d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:04.104002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea09b-b1c5-4440-98e1-6114fe4837d7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:04.104485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea09b-b1c5-4440-98e1-6114fe4837d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:04.104949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea09b-b1c5-4440-98e1-6114fe4837d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:04.105972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ea09b-b1c5-4440-98e1-6114fe4837d7 None admin] RBAC: Adding query filter params (role.id=1f60cad77a004587a328e9dff0cae088, scope.project.id=8087b41e58fb4aca97cbfced2c011450, user.id=9c1646303f474283bd07c9c93a1887d8) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:04.106520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ea09b-b1c5-4440-98e1-6114fe4837d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:04.108162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90ea09b-b1c5-4440-98e1-6114fe4837d7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:04.114619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 63/130] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:03 2022] GET /identity/v3/role_assignments?user.id=9c1646303f474283bd07c9c93a1887d8&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=1f60cad77a004587a328e9dff0cae088 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:04.115732 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ea09b-b1c5-4440-98e1-6114fe4837d7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:04.268741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bd78b1-02eb-44f2-b2eb-dc566eb0bcce None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:04.333056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ea09b-b1c5-4440-98e1-6114fe4837d7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:04.352645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39e95c-f456-448a-a3bb-8bd6307da28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:04.353366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39e95c-f456-448a-a3bb-8bd6307da28c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:04.353591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39e95c-f456-448a-a3bb-8bd6307da28c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:04.353867 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e39e95c-f456-448a-a3bb-8bd6307da28c None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:04.354545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39e95c-f456-448a-a3bb-8bd6307da28c None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:04.354854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39e95c-f456-448a-a3bb-8bd6307da28c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:04.356275 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e39e95c-f456-448a-a3bb-8bd6307da28c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:04.374676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 68/131] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:04.379819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39e95c-f456-448a-a3bb-8bd6307da28c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:04.385457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa527b9-d8d4-4fb1-8cd9-75dd201c5690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:04.386118 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa527b9-d8d4-4fb1-8cd9-75dd201c5690 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:04.386396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa527b9-d8d4-4fb1-8cd9-75dd201c5690 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:04.386672 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa527b9-d8d4-4fb1-8cd9-75dd201c5690 None admin] PATH_INFO: `/v3/users/68d47cfe578c4b96a86d50549ee2c4ed` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:04.391125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa527b9-d8d4-4fb1-8cd9-75dd201c5690 None admin] RBAC: Authorizing `identity:get_user(user_id=68d47cfe578c4b96a86d50549ee2c4ed)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:04.393135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa527b9-d8d4-4fb1-8cd9-75dd201c5690 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:04.395520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 64/132] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:04 2022] GET /identity/v3/users/68d47cfe578c4b96a86d50549ee2c4ed => generated 275 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:04.404382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa527b9-d8d4-4fb1-8cd9-75dd201c5690 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:04.477168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:04.481467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:04.568094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa527b9-d8d4-4fb1-8cd9-75dd201c5690 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:04.596204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e39e95c-f456-448a-a3bb-8bd6307da28c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:04.626069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd7c03-642b-4863-b84c-076365c1d923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:04.626636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd7c03-642b-4863-b84c-076365c1d923 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:04.626938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd7c03-642b-4863-b84c-076365c1d923 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:04.627213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd7c03-642b-4863-b84c-076365c1d923 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:04.628642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd7c03-642b-4863-b84c-076365c1d923 None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:04.630136 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd7c03-642b-4863-b84c-076365c1d923 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:04.631762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 65/133] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:04 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:04.639467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd7c03-642b-4863-b84c-076365c1d923 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:04.671884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9578ec-1697-4bae-85ef-b04145d1bd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:04.672641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9578ec-1697-4bae-85ef-b04145d1bd7b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:04.672925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9578ec-1697-4bae-85ef-b04145d1bd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:04.673264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9578ec-1697-4bae-85ef-b04145d1bd7b None admin] PATH_INFO: `/v3/users/68d47cfe578c4b96a86d50549ee2c4ed` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:04.674438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9578ec-1697-4bae-85ef-b04145d1bd7b None admin] RBAC: Authorizing `identity:get_user(user_id=68d47cfe578c4b96a86d50549ee2c4ed)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:04.675895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9578ec-1697-4bae-85ef-b04145d1bd7b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:04.677520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 69/134] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:04 2022] GET /identity/v3/users/68d47cfe578c4b96a86d50549ee2c4ed => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:04.683226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9578ec-1697-4bae-85ef-b04145d1bd7b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:04.821860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd7c03-642b-4863-b84c-076365c1d923 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:04.835418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:04.885634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e788f2-570a-4960-bb98-3c262a846d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:04.886219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e788f2-570a-4960-bb98-3c262a846d1e None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:04.886633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e788f2-570a-4960-bb98-3c262a846d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:04.886931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e788f2-570a-4960-bb98-3c262a846d1e None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450/users/68d47cfe578c4b96a86d50549ee2c4ed/roles/dea086603b04451680a14ae806205d79` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:04.891393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e788f2-570a-4960-bb98-3c262a846d1e None admin] RBAC: Authorizing `identity:create_grant(project_id=8087b41e58fb4aca97cbfced2c011450, user_id=68d47cfe578c4b96a86d50549ee2c4ed, role_id=dea086603b04451680a14ae806205d79)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:04.893176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e788f2-570a-4960-bb98-3c262a846d1e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:04.909361 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9578ec-1697-4bae-85ef-b04145d1bd7b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:04.916494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 66/135] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:56:04 2022] PUT /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450/users/68d47cfe578c4b96a86d50549ee2c4ed/roles/dea086603b04451680a14ae806205d79 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 21:56:04.938901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:04.964195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4d540f-7689-43ac-b092-4b7d40f752b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:04.964847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d540f-7689-43ac-b092-4b7d40f752b2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:04.965146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d540f-7689-43ac-b092-4b7d40f752b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:04.965427 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4d540f-7689-43ac-b092-4b7d40f752b2 None admin] PATH_INFO: `/v3/projects/8d8a80f2366945e4836c5b2979ae10fc` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:04.969911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d540f-7689-43ac-b092-4b7d40f752b2 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8a80f2366945e4836c5b2979ae10fc)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:04.971404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4d540f-7689-43ac-b092-4b7d40f752b2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:04.973232 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 70/136] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:04 2022] GET /identity/v3/projects/8d8a80f2366945e4836c5b2979ae10fc => generated 312 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:04.979436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e788f2-570a-4960-bb98-3c262a846d1e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:05.120510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e788f2-570a-4960-bb98-3c262a846d1e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:05.163840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde92c11-e027-42dc-94e9-9563d45c8f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:05.164454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde92c11-e027-42dc-94e9-9563d45c8f77 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:05.164736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde92c11-e027-42dc-94e9-9563d45c8f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:05.165014 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde92c11-e027-42dc-94e9-9563d45c8f77 None admin] PATH_INFO: `/v3/projects/8d8a80f2366945e4836c5b2979ae10fc/users/68d47cfe578c4b96a86d50549ee2c4ed/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:05.167080 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde92c11-e027-42dc-94e9-9563d45c8f77 None admin] RBAC: Authorizing `identity:create_grant(project_id=8d8a80f2366945e4836c5b2979ae10fc, user_id=68d47cfe578c4b96a86d50549ee2c4ed, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:05.168564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde92c11-e027-42dc-94e9-9563d45c8f77 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:05.185185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 67/137] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:56:04 2022] PUT /identity/v3/projects/8d8a80f2366945e4836c5b2979ae10fc/users/68d47cfe578c4b96a86d50549ee2c4ed/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:05.928696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49903d7-56c5-452a-bee9-c793b4275a96 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:05.929251 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49903d7-56c5-452a-bee9-c793b4275a96 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:05.929251 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49903d7-56c5-452a-bee9-c793b4275a96 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:05.930897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 71/138] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:05.936367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36f500-349b-4942-a95a-32ffafc3a175 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:05.937358 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36f500-349b-4942-a95a-32ffafc3a175 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:05.937548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a36f500-349b-4942-a95a-32ffafc3a175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:05.959766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3a36f500-349b-4942-a95a-32ffafc3a175 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:05.960979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3a36f500-349b-4942-a95a-32ffafc3a175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:05.995174 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:06.059053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 68/139] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:06.443506 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d3b91-8b03-46c2-a7e2-beade79af400 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:06.443876 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d3b91-8b03-46c2-a7e2-beade79af400 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:06.444074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d3b91-8b03-46c2-a7e2-beade79af400 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:06.445582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 72/140] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:06.450956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8058e34-ca7c-4451-9ec9-ac3130379314 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:06.451481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8058e34-ca7c-4451-9ec9-ac3130379314 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:06.451949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8058e34-ca7c-4451-9ec9-ac3130379314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:06.472681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e8058e34-ca7c-4451-9ec9-ac3130379314 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:06.474208 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e8058e34-ca7c-4451-9ec9-ac3130379314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:06.509271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d3b91-8b03-46c2-a7e2-beade79af400 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:06.548227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 69/141] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:06.646344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d3b91-8b03-46c2-a7e2-beade79af400 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:06.681623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa23ea8-982b-46b8-8945-40f3da5ccc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:06.682252 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa23ea8-982b-46b8-8945-40f3da5ccc4b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:06.682525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa23ea8-982b-46b8-8945-40f3da5ccc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:06.682794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa23ea8-982b-46b8-8945-40f3da5ccc4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:06.683902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa23ea8-982b-46b8-8945-40f3da5ccc4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:06.685442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa23ea8-982b-46b8-8945-40f3da5ccc4b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:06.688034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 73/142] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:06.693840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8058e34-ca7c-4451-9ec9-ac3130379314 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:06.850611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:06.851399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa23ea8-982b-46b8-8945-40f3da5ccc4b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:06.865358 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8058e34-ca7c-4451-9ec9-ac3130379314 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:06.924126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c0987-af73-4d58-add9-116b584333a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:06.925302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c0987-af73-4d58-add9-116b584333a1 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:06.927980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c0987-af73-4d58-add9-116b584333a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:06.928730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c0987-af73-4d58-add9-116b584333a1 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:06.930389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c0987-af73-4d58-add9-116b584333a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:06.932299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c0987-af73-4d58-add9-116b584333a1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:06.964241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 70/143] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:56:06 2022] POST /identity/v3/users => generated 283 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:56:06.979046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:07.003460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e5cfe-c6c6-4431-b104-570e7bf0ffdb None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:07.004308 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e5cfe-c6c6-4431-b104-570e7bf0ffdb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:07.004798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e5cfe-c6c6-4431-b104-570e7bf0ffdb None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:07.006549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 71/144] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:07.018242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae80bc2-a930-462c-a62b-4cfbd59051f0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:07.018713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae80bc2-a930-462c-a62b-4cfbd59051f0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:07.019065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae80bc2-a930-462c-a62b-4cfbd59051f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:07.056025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7ae80bc2-a930-462c-a62b-4cfbd59051f0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:07.060760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7ae80bc2-a930-462c-a62b-4cfbd59051f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:07.064921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa23ea8-982b-46b8-8945-40f3da5ccc4b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:07.143370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b4394-9d18-4300-b128-20e72e1a742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:07.144072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4394-9d18-4300-b128-20e72e1a742a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:07.144352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4394-9d18-4300-b128-20e72e1a742a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:07.144623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b4394-9d18-4300-b128-20e72e1a742a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:07.166454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b4394-9d18-4300-b128-20e72e1a742a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e16b4394-9d18-4300-b128-20e72e1a742a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:07.175298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:07.176249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:07.177010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 74/145] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:56:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:07.182581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b4394-9d18-4300-b128-20e72e1a742a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:07.206535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 72/146] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:07.246032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd03d42-0b1b-4ac0-94d8-2ddf4b758cd9 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:07.246307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd03d42-0b1b-4ac0-94d8-2ddf4b758cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:07.246498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd03d42-0b1b-4ac0-94d8-2ddf4b758cd9 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:07.248120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 73/147] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:07.256120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180bba6-0454-4e65-9b03-76c2ac1ab926 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:07.256554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180bba6-0454-4e65-9b03-76c2ac1ab926 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:07.256922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180bba6-0454-4e65-9b03-76c2ac1ab926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:07.279681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3180bba6-0454-4e65-9b03-76c2ac1ab926 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:07.281961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3180bba6-0454-4e65-9b03-76c2ac1ab926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:07.379312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b4394-9d18-4300-b128-20e72e1a742a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:07.410567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 74/148] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:07.411168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:07.439647 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dca4d6-e36c-49b5-b3ec-ddf858ba7493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:07.440259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dca4d6-e36c-49b5-b3ec-ddf858ba7493 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:07.440534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dca4d6-e36c-49b5-b3ec-ddf858ba7493 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:07.440803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dca4d6-e36c-49b5-b3ec-ddf858ba7493 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:07.441490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dca4d6-e36c-49b5-b3ec-ddf858ba7493 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:07.441790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dca4d6-e36c-49b5-b3ec-ddf858ba7493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:07.443162 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dca4d6-e36c-49b5-b3ec-ddf858ba7493 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:07.451899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 75/149] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:07.458050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180bba6-0454-4e65-9b03-76c2ac1ab926 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:07.462161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:07.589303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dca4d6-e36c-49b5-b3ec-ddf858ba7493 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:07.589517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180bba6-0454-4e65-9b03-76c2ac1ab926 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:07.630271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e07804-a617-46db-88e3-d347983ebd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:07.630964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e07804-a617-46db-88e3-d347983ebd54 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:07.631239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e07804-a617-46db-88e3-d347983ebd54 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:07.631511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e07804-a617-46db-88e3-d347983ebd54 None admin] PATH_INFO: `/v3/users/9a8dd819f6d742ceba5a020c855827f8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:07.632785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e07804-a617-46db-88e3-d347983ebd54 None admin] RBAC: Authorizing `identity:get_user(user_id=9a8dd819f6d742ceba5a020c855827f8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:07.634262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e07804-a617-46db-88e3-d347983ebd54 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:07.636126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 75/150] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:07 2022] GET /identity/v3/users/9a8dd819f6d742ceba5a020c855827f8 => generated 247 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:07.646286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e07804-a617-46db-88e3-d347983ebd54 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:07.772945 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dca4d6-e36c-49b5-b3ec-ddf858ba7493 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:07.787093 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:07.827779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e07804-a617-46db-88e3-d347983ebd54 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:07.833857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d28f4-2f7b-4a69-bdb1-ef3d2aa38d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:07.834553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d28f4-2f7b-4a69-bdb1-ef3d2aa38d18 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:07.834834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d28f4-2f7b-4a69-bdb1-ef3d2aa38d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:07.835107 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820d28f4-2f7b-4a69-bdb1-ef3d2aa38d18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:07.854640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820d28f4-2f7b-4a69-bdb1-ef3d2aa38d18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-820d28f4-2f7b-4a69-bdb1-ef3d2aa38d18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:07.862299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:07.863146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:07.863894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:07.866438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 76/151] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:07.867564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d28f4-2f7b-4a69-bdb1-ef3d2aa38d18 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:07.874660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176073d-c1ac-4ea9-83cd-685efefd033b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:07.875399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176073d-c1ac-4ea9-83cd-685efefd033b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:07.875720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176073d-c1ac-4ea9-83cd-685efefd033b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:07.876051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176073d-c1ac-4ea9-83cd-685efefd033b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:07.916505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:07.966349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176073d-c1ac-4ea9-83cd-685efefd033b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:07.967855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176073d-c1ac-4ea9-83cd-685efefd033b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:08.011448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820d28f4-2f7b-4a69-bdb1-ef3d2aa38d18 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:08.075532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:08.085823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b5602-0b65-4209-be51-b04e911c2a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:08.086620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b5602-0b65-4209-be51-b04e911c2a11 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:08.086730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b5602-0b65-4209-be51-b04e911c2a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:08.086995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b5602-0b65-4209-be51-b04e911c2a11 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:08.113038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 76/152] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:08.114214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176073d-c1ac-4ea9-83cd-685efefd033b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:08.122246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b5602-0b65-4209-be51-b04e911c2a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-1a3b5602-0b65-4209-be51-b04e911c2a11 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:08.128626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:08.129513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:08.130278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 77/153] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:08.131755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b5602-0b65-4209-be51-b04e911c2a11 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:08.246526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176073d-c1ac-4ea9-83cd-685efefd033b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:08.305877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143d2cb-b8d3-4288-a17a-559b6a0ab080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:08.306867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143d2cb-b8d3-4288-a17a-559b6a0ab080 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:08.307370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143d2cb-b8d3-4288-a17a-559b6a0ab080 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:08.307478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:08.307940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a143d2cb-b8d3-4288-a17a-559b6a0ab080 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:08.309216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143d2cb-b8d3-4288-a17a-559b6a0ab080 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:08.309737 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143d2cb-b8d3-4288-a17a-559b6a0ab080 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:08.311269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a143d2cb-b8d3-4288-a17a-559b6a0ab080 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:08.315650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b5602-0b65-4209-be51-b04e911c2a11 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:08.324367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 77/154] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:08.325459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143d2cb-b8d3-4288-a17a-559b6a0ab080 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:08.371044 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b14f0-0671-430d-b669-732c9d43f415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:08.371608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b14f0-0671-430d-b669-732c9d43f415 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:08.371887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b14f0-0671-430d-b669-732c9d43f415 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:08.372159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147b14f0-0671-430d-b669-732c9d43f415 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:08.373350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b14f0-0671-430d-b669-732c9d43f415 None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:08.374886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147b14f0-0671-430d-b669-732c9d43f415 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:08.376527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 78/155] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:08 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:08.377327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b14f0-0671-430d-b669-732c9d43f415 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:08.381028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:08.458498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a143d2cb-b8d3-4288-a17a-559b6a0ab080 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:08.493049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147b14f0-0671-430d-b669-732c9d43f415 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:08.501201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db77f5-bb20-4c0d-b387-ad988867c91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:08.502241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db77f5-bb20-4c0d-b387-ad988867c91c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:08.502692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db77f5-bb20-4c0d-b387-ad988867c91c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:08.503160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db77f5-bb20-4c0d-b387-ad988867c91c None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:08.504019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db77f5-bb20-4c0d-b387-ad988867c91c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:08.504611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db77f5-bb20-4c0d-b387-ad988867c91c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:08.506259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db77f5-bb20-4c0d-b387-ad988867c91c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:08.516578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 78/156] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:08.518347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f1102-f136-42d6-88cf-727dd287e971 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:08.518872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f1102-f136-42d6-88cf-727dd287e971 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:08.519315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f1102-f136-42d6-88cf-727dd287e971 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:08.520921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 79/157] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:08.522063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f1102-f136-42d6-88cf-727dd287e971 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:08.537738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821c3df-0976-46c3-9e0b-8256eb73dd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:08.538387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821c3df-0976-46c3-9e0b-8256eb73dd20 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:08.539180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821c3df-0976-46c3-9e0b-8256eb73dd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:08.539180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e821c3df-0976-46c3-9e0b-8256eb73dd20 None admin] PATH_INFO: `/v3/users/68d47cfe578c4b96a86d50549ee2c4ed` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:08.540805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821c3df-0976-46c3-9e0b-8256eb73dd20 None admin] RBAC: Authorizing `identity:get_user(user_id=68d47cfe578c4b96a86d50549ee2c4ed)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:08.542360 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e821c3df-0976-46c3-9e0b-8256eb73dd20 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:08.544384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 79/158] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:08 2022] GET /identity/v3/users/68d47cfe578c4b96a86d50549ee2c4ed => generated 275 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:08.545344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821c3df-0976-46c3-9e0b-8256eb73dd20 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:08.583824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:08.642718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:08.651597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f1102-f136-42d6-88cf-727dd287e971 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:08.659792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e821c3df-0976-46c3-9e0b-8256eb73dd20 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:08.697306 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b3d50-23c7-48c6-892c-dab2f3334cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:08.697900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b3d50-23c7-48c6-892c-dab2f3334cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:08.698182 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b3d50-23c7-48c6-892c-dab2f3334cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:08.698457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b3d50-23c7-48c6-892c-dab2f3334cb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:08.699184 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b3d50-23c7-48c6-892c-dab2f3334cb3 None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=8087b41e58fb4aca97cbfced2c011450, user.id=9a8dd819f6d742ceba5a020c855827f8) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:08.699497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b3d50-23c7-48c6-892c-dab2f3334cb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:08.700959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b3d50-23c7-48c6-892c-dab2f3334cb3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:08.706316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0ab6e-affb-4d73-906d-98f8726959b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:08.706973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ab6e-affb-4d73-906d-98f8726959b6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:08.707251 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ab6e-affb-4d73-906d-98f8726959b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:08.707521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0ab6e-affb-4d73-906d-98f8726959b6 None admin] PATH_INFO: `/v3/users/9c1646303f474283bd07c9c93a1887d8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:08.712038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 80/159] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:08 2022] GET /identity/v3/role_assignments?user.id=9a8dd819f6d742ceba5a020c855827f8&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:08.712198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0ab6e-affb-4d73-906d-98f8726959b6 None admin] RBAC: Authorizing `identity:get_user(user_id=9c1646303f474283bd07c9c93a1887d8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:08.712291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f77ae-5253-46a9-8395-f9ae4877cfc3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:08.712291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f77ae-5253-46a9-8395-f9ae4877cfc3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:08.712291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f77ae-5253-46a9-8395-f9ae4877cfc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:08.712471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0ab6e-affb-4d73-906d-98f8726959b6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:08.712471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 80/160] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:08 2022] GET /identity/v3/users/9c1646303f474283bd07c9c93a1887d8 => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:08.712471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0ab6e-affb-4d73-906d-98f8726959b6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:08.737454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c32f77ae-5253-46a9-8395-f9ae4877cfc3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:08.739375 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c32f77ae-5253-46a9-8395-f9ae4877cfc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:08.852397 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 81/161] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:08.854262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6fd0c-4d36-46cf-9f1f-8e2541d469fb None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:08.854792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6fd0c-4d36-46cf-9f1f-8e2541d469fb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:08.855296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc6fd0c-4d36-46cf-9f1f-8e2541d469fb None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:08.856905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 82/162] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:08.857988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6fd0c-4d36-46cf-9f1f-8e2541d469fb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:08.883723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:08.887169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0ab6e-affb-4d73-906d-98f8726959b6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:08.946733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4488605-859b-4f6c-a916-5ebbd34431c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:08.947260 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4488605-859b-4f6c-a916-5ebbd34431c5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:08.947584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4488605-859b-4f6c-a916-5ebbd34431c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:08.947916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4488605-859b-4f6c-a916-5ebbd34431c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:08.969158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:08.997879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc6fd0c-4d36-46cf-9f1f-8e2541d469fb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:09.078750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4488605-859b-4f6c-a916-5ebbd34431c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:09.080807 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4488605-859b-4f6c-a916-5ebbd34431c5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:09.106771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95538d62-69ec-4ca0-a2c3-a2de453e74ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:09.107413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95538d62-69ec-4ca0-a2c3-a2de453e74ae None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:09.107695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95538d62-69ec-4ca0-a2c3-a2de453e74ae None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:09.107969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95538d62-69ec-4ca0-a2c3-a2de453e74ae None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:09.113879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95538d62-69ec-4ca0-a2c3-a2de453e74ae None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:09.115353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95538d62-69ec-4ca0-a2c3-a2de453e74ae None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:09.117058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 83/163] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:08 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:09.119372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d7392-d6cb-4756-b1df-97c9b8355ab7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:09.119372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d7392-d6cb-4756-b1df-97c9b8355ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:09.119372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d7392-d6cb-4756-b1df-97c9b8355ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:09.154120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ed1d7392-d6cb-4756-b1df-97c9b8355ab7 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:09.160018 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ed1d7392-d6cb-4756-b1df-97c9b8355ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:09.250535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:09.254049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 81/164] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:09.254962 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4488605-859b-4f6c-a916-5ebbd34431c5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:09.331870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 84/165] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:09.332580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d7392-d6cb-4756-b1df-97c9b8355ab7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:09.433006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4488605-859b-4f6c-a916-5ebbd34431c5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:09.481411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:09.502028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d7392-d6cb-4756-b1df-97c9b8355ab7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:09.531510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dfb01-b2f8-4d7f-af08-b1ef19b0313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:09.532196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dfb01-b2f8-4d7f-af08-b1ef19b0313a None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:09.532475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dfb01-b2f8-4d7f-af08-b1ef19b0313a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:09.532753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780dfb01-b2f8-4d7f-af08-b1ef19b0313a None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450/users/9c1646303f474283bd07c9c93a1887d8/roles/1f60cad77a004587a328e9dff0cae088` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:09.538464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780dfb01-b2f8-4d7f-af08-b1ef19b0313a None admin] RBAC: Authorizing `identity:create_grant(project_id=8087b41e58fb4aca97cbfced2c011450, user_id=9c1646303f474283bd07c9c93a1887d8, role_id=1f60cad77a004587a328e9dff0cae088)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:09.539933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780dfb01-b2f8-4d7f-af08-b1ef19b0313a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:09.569832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028553f0-c485-4346-8760-63cb628134ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:09.570492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028553f0-c485-4346-8760-63cb628134ca None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:09.570762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028553f0-c485-4346-8760-63cb628134ca None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:09.571093 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028553f0-c485-4346-8760-63cb628134ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:09.579709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 82/166] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:56:09 2022] PUT /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450/users/9c1646303f474283bd07c9c93a1887d8/roles/1f60cad77a004587a328e9dff0cae088 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:09.580572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dfb01-b2f8-4d7f-af08-b1ef19b0313a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:09.584690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028553f0-c485-4346-8760-63cb628134ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-028553f0-c485-4346-8760-63cb628134ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:09.589625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:09.590562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:09.591553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 85/167] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:09.604992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028553f0-c485-4346-8760-63cb628134ca None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:09.710288 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:09.781759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028553f0-c485-4346-8760-63cb628134ca None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:09.800737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780dfb01-b2f8-4d7f-af08-b1ef19b0313a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:09.841971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b90df-5132-4a26-8eae-8591ddfc060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:09.842721 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b90df-5132-4a26-8eae-8591ddfc060a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:09.843714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b90df-5132-4a26-8eae-8591ddfc060a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:09.843987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b90df-5132-4a26-8eae-8591ddfc060a None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:09.844859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b90df-5132-4a26-8eae-8591ddfc060a None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:09.845191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b90df-5132-4a26-8eae-8591ddfc060a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:09.846626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b90df-5132-4a26-8eae-8591ddfc060a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:09.867568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 86/168] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:09.868462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b90df-5132-4a26-8eae-8591ddfc060a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:09.885863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d2d7f-f81a-4620-9f40-dadfb030a81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:09.886513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d2d7f-f81a-4620-9f40-dadfb030a81b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:09.886790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d2d7f-f81a-4620-9f40-dadfb030a81b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:09.887066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d2d7f-f81a-4620-9f40-dadfb030a81b None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:09.890457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d2d7f-f81a-4620-9f40-dadfb030a81b None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:09.892014 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d2d7f-f81a-4620-9f40-dadfb030a81b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:09.893820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 83/169] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:09 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:09.894359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:09.899909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d2d7f-f81a-4620-9f40-dadfb030a81b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:10.052067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d2d7f-f81a-4620-9f40-dadfb030a81b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:10.083823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:10.095235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63675b76-02b8-4453-91bc-eb21fb2241e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:10.096245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63675b76-02b8-4453-91bc-eb21fb2241e2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:10.096529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63675b76-02b8-4453-91bc-eb21fb2241e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:10.096806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63675b76-02b8-4453-91bc-eb21fb2241e2 None admin] PATH_INFO: `/v3/users/68d47cfe578c4b96a86d50549ee2c4ed` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:10.098003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63675b76-02b8-4453-91bc-eb21fb2241e2 None admin] RBAC: Authorizing `identity:get_user(user_id=68d47cfe578c4b96a86d50549ee2c4ed)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:10.099466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63675b76-02b8-4453-91bc-eb21fb2241e2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:10.101480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 84/170] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:09 2022] GET /identity/v3/users/68d47cfe578c4b96a86d50549ee2c4ed => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:10.103118 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b90df-5132-4a26-8eae-8591ddfc060a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:10.105591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63675b76-02b8-4453-91bc-eb21fb2241e2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:10.193832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b7f54-f608-4711-8713-df9a5c8d96e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:10.194476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b7f54-f608-4711-8713-df9a5c8d96e6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:10.194756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b7f54-f608-4711-8713-df9a5c8d96e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:10.195025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7b7f54-f608-4711-8713-df9a5c8d96e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:10.222757 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7b7f54-f608-4711-8713-df9a5c8d96e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9a7b7f54-f608-4711-8713-df9a5c8d96e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:10.229220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:10.230192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:10.230971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 87/171] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:10.232482 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63675b76-02b8-4453-91bc-eb21fb2241e2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:10.235947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b7f54-f608-4711-8713-df9a5c8d96e6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:10.295148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2845eb-19bd-4c40-a358-51fa4e5c66fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:10.296782 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2845eb-19bd-4c40-a358-51fa4e5c66fc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:10.297069 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2845eb-19bd-4c40-a358-51fa4e5c66fc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:10.298889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2845eb-19bd-4c40-a358-51fa4e5c66fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:10.299635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2845eb-19bd-4c40-a358-51fa4e5c66fc None admin] RBAC: Adding query filter params (role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, scope.project.id=8087b41e58fb4aca97cbfced2c011450, user.id=68d47cfe578c4b96a86d50549ee2c4ed) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:10.299950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2845eb-19bd-4c40-a358-51fa4e5c66fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:10.303702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2845eb-19bd-4c40-a358-51fa4e5c66fc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:10.318240 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 85/172] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:10 2022] GET /identity/v3/role_assignments?user.id=68d47cfe578c4b96a86d50549ee2c4ed&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:10.319161 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2845eb-19bd-4c40-a358-51fa4e5c66fc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:10.378235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:10.413059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7b7f54-f608-4711-8713-df9a5c8d96e6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:10.453347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:10.454315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336a08a-e83c-486e-aa11-85ec6594eede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:10.456103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336a08a-e83c-486e-aa11-85ec6594eede None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:10.456103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336a08a-e83c-486e-aa11-85ec6594eede None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:10.456103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336a08a-e83c-486e-aa11-85ec6594eede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:10.532625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336a08a-e83c-486e-aa11-85ec6594eede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:10.534129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336a08a-e83c-486e-aa11-85ec6594eede None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:10.540900 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2845eb-19bd-4c40-a358-51fa4e5c66fc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:10.620530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75f3ad-d042-48a9-98b4-734213c97000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:10.621067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75f3ad-d042-48a9-98b4-734213c97000 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:10.621374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75f3ad-d042-48a9-98b4-734213c97000 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:10.621650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75f3ad-d042-48a9-98b4-734213c97000 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:10.622321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75f3ad-d042-48a9-98b4-734213c97000 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:10.622629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75f3ad-d042-48a9-98b4-734213c97000 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:10.625428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75f3ad-d042-48a9-98b4-734213c97000 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:10.641497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:10.647249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 86/173] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:10.658199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75f3ad-d042-48a9-98b4-734213c97000 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:10.671691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 88/174] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:10.678300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336a08a-e83c-486e-aa11-85ec6594eede None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:10.808051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336a08a-e83c-486e-aa11-85ec6594eede None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:10.851904 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b0301-ab3b-449d-8f51-d38746b7bfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:10.852549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b0301-ab3b-449d-8f51-d38746b7bfbf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:10.852831 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b0301-ab3b-449d-8f51-d38746b7bfbf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:10.853123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b0301-ab3b-449d-8f51-d38746b7bfbf None admin] PATH_INFO: `/v3/projects/8d8a80f2366945e4836c5b2979ae10fc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:10.854432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b0301-ab3b-449d-8f51-d38746b7bfbf None admin] RBAC: Authorizing `identity:get_project(project_id=8d8a80f2366945e4836c5b2979ae10fc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:10.855976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b0301-ab3b-449d-8f51-d38746b7bfbf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:10.861851 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 89/175] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:10 2022] GET /identity/v3/projects/8d8a80f2366945e4836c5b2979ae10fc => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:10.868096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b0301-ab3b-449d-8f51-d38746b7bfbf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:10.941552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:10.959480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75f3ad-d042-48a9-98b4-734213c97000 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:10.992792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:11.033891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a33edb-f527-46c9-a207-c73ff3f9f463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:11.034546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a33edb-f527-46c9-a207-c73ff3f9f463 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:11.034828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a33edb-f527-46c9-a207-c73ff3f9f463 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:11.035100 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a33edb-f527-46c9-a207-c73ff3f9f463 None admin] PATH_INFO: `/v3/users/68d47cfe578c4b96a86d50549ee2c4ed` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:11.037297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b0301-ab3b-449d-8f51-d38746b7bfbf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:11.041949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a33edb-f527-46c9-a207-c73ff3f9f463 None admin] RBAC: Authorizing `identity:get_user(user_id=68d47cfe578c4b96a86d50549ee2c4ed)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:11.043432 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a33edb-f527-46c9-a207-c73ff3f9f463 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:11.045464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 87/176] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:10 2022] GET /identity/v3/users/68d47cfe578c4b96a86d50549ee2c4ed => generated 275 bytes in 388 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:11.056468 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a33edb-f527-46c9-a207-c73ff3f9f463 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:11.140138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0fd34-140d-40dc-99c8-d96648149cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:11.140836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0fd34-140d-40dc-99c8-d96648149cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:11.141032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0fd34-140d-40dc-99c8-d96648149cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:11.141331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0fd34-140d-40dc-99c8-d96648149cb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:11.142067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0fd34-140d-40dc-99c8-d96648149cb6 None admin] RBAC: Adding query filter params (role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, scope.project.id=8d8a80f2366945e4836c5b2979ae10fc, user.id=68d47cfe578c4b96a86d50549ee2c4ed) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:11.142375 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0fd34-140d-40dc-99c8-d96648149cb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:11.143832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0fd34-140d-40dc-99c8-d96648149cb6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:11.161987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 90/177] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:10 2022] GET /identity/v3/role_assignments?user.id=68d47cfe578c4b96a86d50549ee2c4ed&scope.project.id=8d8a80f2366945e4836c5b2979ae10fc&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:11.231741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a33edb-f527-46c9-a207-c73ff3f9f463 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:11.276135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b7cc0-f830-4152-9d09-daa57ecbe5f5 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:11.276448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b7cc0-f830-4152-9d09-daa57ecbe5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:11.276690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64b7cc0-f830-4152-9d09-daa57ecbe5f5 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:11.278083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 91/178] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:11.278788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2227c-25ea-4d59-a961-20c834d1e78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:11.279407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2227c-25ea-4d59-a961-20c834d1e78c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:11.279683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2227c-25ea-4d59-a961-20c834d1e78c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:11.279952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2227c-25ea-4d59-a961-20c834d1e78c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:11.283898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af207073-f989-47e5-8613-9ae534ec7fc6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:11.284173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af207073-f989-47e5-8613-9ae534ec7fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:11.284435 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af207073-f989-47e5-8613-9ae534ec7fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:11.325865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-af207073-f989-47e5-8613-9ae534ec7fc6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:11.327266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-af207073-f989-47e5-8613-9ae534ec7fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:11.418280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:11.440845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2227c-25ea-4d59-a961-20c834d1e78c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:11.442801 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2227c-25ea-4d59-a961-20c834d1e78c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:11.465614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 92/179] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:11.572940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 88/180] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:11.581556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af207073-f989-47e5-8613-9ae534ec7fc6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:11.636929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:11.715165 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af207073-f989-47e5-8613-9ae534ec7fc6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:11.751892 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f642f2db-d704-4c4a-a932-6e2a35e9efe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:11.752600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642f2db-d704-4c4a-a932-6e2a35e9efe4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:11.752887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642f2db-d704-4c4a-a932-6e2a35e9efe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:11.753203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f642f2db-d704-4c4a-a932-6e2a35e9efe4 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:11.754346 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642f2db-d704-4c4a-a932-6e2a35e9efe4 None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:11.755789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f642f2db-d704-4c4a-a932-6e2a35e9efe4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:11.757633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 93/181] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:11 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:11.763531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2227c-25ea-4d59-a961-20c834d1e78c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:11.797899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c17e60-bc28-4465-8167-b5c9ed7cc02a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:11.798177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c17e60-bc28-4465-8167-b5c9ed7cc02a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:11.798447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c17e60-bc28-4465-8167-b5c9ed7cc02a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:11.800153 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 94/182] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:11.806258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97badf-d94f-4e7d-8bbf-7ad2a25787a8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:11.806722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97badf-d94f-4e7d-8bbf-7ad2a25787a8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:11.807141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f97badf-d94f-4e7d-8bbf-7ad2a25787a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:11.826624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4f97badf-d94f-4e7d-8bbf-7ad2a25787a8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:11.828302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4f97badf-d94f-4e7d-8bbf-7ad2a25787a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:11.905264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 95/183] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:11.906462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97badf-d94f-4e7d-8bbf-7ad2a25787a8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:11.937799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:11.954924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2227c-25ea-4d59-a961-20c834d1e78c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:11.964416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:12.015399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21373b4-01ab-4ac6-b41a-3cb56ea87956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:12.016006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21373b4-01ab-4ac6-b41a-3cb56ea87956 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:12.016282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21373b4-01ab-4ac6-b41a-3cb56ea87956 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:12.016554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21373b4-01ab-4ac6-b41a-3cb56ea87956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:12.017297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21373b4-01ab-4ac6-b41a-3cb56ea87956 None admin] RBAC: Adding query filter params (role.id=dea086603b04451680a14ae806205d79, scope.project.id=8087b41e58fb4aca97cbfced2c011450, user.id=68d47cfe578c4b96a86d50549ee2c4ed) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:12.017603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21373b4-01ab-4ac6-b41a-3cb56ea87956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:12.019041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21373b4-01ab-4ac6-b41a-3cb56ea87956 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:12.028057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 89/184] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:11 2022] GET /identity/v3/role_assignments?user.id=68d47cfe578c4b96a86d50549ee2c4ed&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=dea086603b04451680a14ae806205d79 => generated 608 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:12.068008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f97badf-d94f-4e7d-8bbf-7ad2a25787a8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:12.115976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c0ba49-c48d-4f5f-b9e3-5af3525d134e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:12.116555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0ba49-c48d-4f5f-b9e3-5af3525d134e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:12.116835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0ba49-c48d-4f5f-b9e3-5af3525d134e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:12.117128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c0ba49-c48d-4f5f-b9e3-5af3525d134e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:12.140580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c0ba49-c48d-4f5f-b9e3-5af3525d134e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a6c0ba49-c48d-4f5f-b9e3-5af3525d134e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:12.145882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:12.146811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:12.147601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 96/185] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:56:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:12.153317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21373b4-01ab-4ac6-b41a-3cb56ea87956 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:12.212497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432470e-1843-4096-923c-f6f370d9af49 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:12.213174 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432470e-1843-4096-923c-f6f370d9af49 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:12.213649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c432470e-1843-4096-923c-f6f370d9af49 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:12.215201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 97/186] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:12.235182 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754c0bd-485a-4299-ad76-e06f5a258e25 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:12.235792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754c0bd-485a-4299-ad76-e06f5a258e25 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:12.236302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754c0bd-485a-4299-ad76-e06f5a258e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:12.262305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8754c0bd-485a-4299-ad76-e06f5a258e25 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:12.263628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8754c0bd-485a-4299-ad76-e06f5a258e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:12.301334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21373b4-01ab-4ac6-b41a-3cb56ea87956 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:12.338652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 98/187] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:12.345128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915de18-b4a4-46e2-808e-d855f4bdda53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:12.345280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754c0bd-485a-4299-ad76-e06f5a258e25 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:12.345358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915de18-b4a4-46e2-808e-d855f4bdda53 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:12.345358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915de18-b4a4-46e2-808e-d855f4bdda53 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:12.345358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915de18-b4a4-46e2-808e-d855f4bdda53 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:12.345358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915de18-b4a4-46e2-808e-d855f4bdda53 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:12.345358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915de18-b4a4-46e2-808e-d855f4bdda53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:12.345358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915de18-b4a4-46e2-808e-d855f4bdda53 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:12.355475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 90/188] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:12.362160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915de18-b4a4-46e2-808e-d855f4bdda53 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:12.367386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:12.390589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:12.475488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754c0bd-485a-4299-ad76-e06f5a258e25 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:12.485182 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915de18-b4a4-46e2-808e-d855f4bdda53 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:12.517459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87b27c-8656-4187-95a0-565d824ea9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:12.518241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87b27c-8656-4187-95a0-565d824ea9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:12.518612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87b27c-8656-4187-95a0-565d824ea9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:12.518976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87b27c-8656-4187-95a0-565d824ea9a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:12.531607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87b27c-8656-4187-95a0-565d824ea9a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:12.533302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0a87b-f327-4299-885c-1d4e72e06437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:12.534094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0a87b-f327-4299-885c-1d4e72e06437 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:12.534541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0a87b-f327-4299-885c-1d4e72e06437 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:12.534960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0a87b-f327-4299-885c-1d4e72e06437 None admin] PATH_INFO: `/v3/users/254c256c63404b3fa858d7b68b45a470` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cc87b27c-8656-4187-95a0-565d824ea9a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:12.536998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:12.538187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:12.539181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:12.540880 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 99/189] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:12.546371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87b27c-8656-4187-95a0-565d824ea9a0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:12.555121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0a87b-f327-4299-885c-1d4e72e06437 None admin] RBAC: Authorizing `identity:get_user(user_id=254c256c63404b3fa858d7b68b45a470)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:12.557360 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0a87b-f327-4299-885c-1d4e72e06437 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:12.565430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 91/190] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:12 2022] GET /identity/v3/users/254c256c63404b3fa858d7b68b45a470 => generated 283 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:12.567125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0a87b-f327-4299-885c-1d4e72e06437 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:12.650658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:12.660625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:12.678201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0a87b-f327-4299-885c-1d4e72e06437 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:12.682937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87b27c-8656-4187-95a0-565d824ea9a0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:12.713926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb63b57-1c20-44cb-b968-811d55ea5f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:12.714538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb63b57-1c20-44cb-b968-811d55ea5f6f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:12.714816 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb63b57-1c20-44cb-b968-811d55ea5f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:12.715089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb63b57-1c20-44cb-b968-811d55ea5f6f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:12.720624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf9a35-2b33-4605-9303-f347ba9e3870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:12.721262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9a35-2b33-4605-9303-f347ba9e3870 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:12.721537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9a35-2b33-4605-9303-f347ba9e3870 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:12.721891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9a35-2b33-4605-9303-f347ba9e3870 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:12.722549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf9a35-2b33-4605-9303-f347ba9e3870 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:12.722867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf9a35-2b33-4605-9303-f347ba9e3870 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:12.724267 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf9a35-2b33-4605-9303-f347ba9e3870 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:12.734155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 100/191] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:12.738195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf9a35-2b33-4605-9303-f347ba9e3870 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:12.792838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb63b57-1c20-44cb-b968-811d55ea5f6f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:12.794316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb63b57-1c20-44cb-b968-811d55ea5f6f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:12.852777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf9a35-2b33-4605-9303-f347ba9e3870 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:12.871835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:12.886039 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c1968-cf4b-412c-b551-ff88b69949ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:12.886685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c1968-cf4b-412c-b551-ff88b69949ff None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:12.886960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c1968-cf4b-412c-b551-ff88b69949ff None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:12.887227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c1968-cf4b-412c-b551-ff88b69949ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:12.888319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c1968-cf4b-412c-b551-ff88b69949ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:12.890344 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23c1968-cf4b-412c-b551-ff88b69949ff None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:12.891899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 101/192] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:12.892648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c1968-cf4b-412c-b551-ff88b69949ff None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:12.900446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 92/193] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:12.902081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c7a8b-91ed-4842-baac-1a218c8f66f6 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:12.902419 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c7a8b-91ed-4842-baac-1a218c8f66f6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:12.902692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c7a8b-91ed-4842-baac-1a218c8f66f6 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:12.903942 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 93/194] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:12.908684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c7a8b-91ed-4842-baac-1a218c8f66f6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:12.909368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:12.994333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c1968-cf4b-412c-b551-ff88b69949ff None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:13.005142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c7a8b-91ed-4842-baac-1a218c8f66f6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:13.027193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877e4d5-e490-4f21-be78-7a5cf95e0fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:13.027812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877e4d5-e490-4f21-be78-7a5cf95e0fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:13.028090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877e4d5-e490-4f21-be78-7a5cf95e0fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:13.028360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d877e4d5-e490-4f21-be78-7a5cf95e0fe1 None admin] PATH_INFO: `/v3/users/254c256c63404b3fa858d7b68b45a470` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:13.029495 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877e4d5-e490-4f21-be78-7a5cf95e0fe1 None admin] RBAC: Authorizing `identity:get_user(user_id=254c256c63404b3fa858d7b68b45a470)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:13.031684 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d877e4d5-e490-4f21-be78-7a5cf95e0fe1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:13.033274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 102/195] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:12 2022] GET /identity/v3/users/254c256c63404b3fa858d7b68b45a470 => generated 283 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:13.038057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877e4d5-e490-4f21-be78-7a5cf95e0fe1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:13.038959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d050d0e-132d-42df-8ba3-78f561876b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:13.039563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050d0e-132d-42df-8ba3-78f561876b69 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:13.039839 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050d0e-132d-42df-8ba3-78f561876b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:13.040111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d050d0e-132d-42df-8ba3-78f561876b69 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:13.040760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d050d0e-132d-42df-8ba3-78f561876b69 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:13.042224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d050d0e-132d-42df-8ba3-78f561876b69 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:13.064638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 94/196] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:56:12 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:56:13.065879 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec72ee7-d165-498e-97b5-dd22f373f96c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:13.066161 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec72ee7-d165-498e-97b5-dd22f373f96c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:13.066426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec72ee7-d165-498e-97b5-dd22f373f96c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:13.086669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1ec72ee7-d165-498e-97b5-dd22f373f96c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:13.087884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1ec72ee7-d165-498e-97b5-dd22f373f96c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:13.113577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:13.143697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:13.162645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 95/197] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:13.163263 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec72ee7-d165-498e-97b5-dd22f373f96c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:13.178435 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d877e4d5-e490-4f21-be78-7a5cf95e0fe1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:13.229434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310659b-03b6-4197-ae7f-78af3ab996ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:13.230728 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310659b-03b6-4197-ae7f-78af3ab996ec None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:13.231751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310659b-03b6-4197-ae7f-78af3ab996ec None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:13.232234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310659b-03b6-4197-ae7f-78af3ab996ec None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:13.234455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310659b-03b6-4197-ae7f-78af3ab996ec None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:13.236584 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310659b-03b6-4197-ae7f-78af3ab996ec None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:13.239127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 103/198] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:13 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:13.245862 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310659b-03b6-4197-ae7f-78af3ab996ec None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:13.295831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec72ee7-d165-498e-97b5-dd22f373f96c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:13.346694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b9eb9-d3a2-4bdb-abf5-b3fd60ba8a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:13.347450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b9eb9-d3a2-4bdb-abf5-b3fd60ba8a33 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:13.347744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b9eb9-d3a2-4bdb-abf5-b3fd60ba8a33 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:13.348014 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b9eb9-d3a2-4bdb-abf5-b3fd60ba8a33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:13.371874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310659b-03b6-4197-ae7f-78af3ab996ec None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:13.385937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:13.421244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47ca00-2686-4e4d-a3e5-d547339f67a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:13.422150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47ca00-2686-4e4d-a3e5-d547339f67a5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:13.422434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47ca00-2686-4e4d-a3e5-d547339f67a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:13.422709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d47ca00-2686-4e4d-a3e5-d547339f67a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:13.423443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47ca00-2686-4e4d-a3e5-d547339f67a5 None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=a36c670998e84f8ba575f12eebf55432, user.id=254c256c63404b3fa858d7b68b45a470) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:13.423754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47ca00-2686-4e4d-a3e5-d547339f67a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:13.425827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d47ca00-2686-4e4d-a3e5-d547339f67a5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:13.434527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b9eb9-d3a2-4bdb-abf5-b3fd60ba8a33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:13.435127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 104/199] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:13 2022] GET /identity/v3/role_assignments?user.id=254c256c63404b3fa858d7b68b45a470&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:13.436392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b9eb9-d3a2-4bdb-abf5-b3fd60ba8a33 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:13.442163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47ca00-2686-4e4d-a3e5-d547339f67a5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:13.460713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:13.572445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 96/200] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:13.577146 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d47ca00-2686-4e4d-a3e5-d547339f67a5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:13.579683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b9eb9-d3a2-4bdb-abf5-b3fd60ba8a33 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:13.619846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d812950-0dda-4b5f-8601-bc0846be335c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:13.620491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d812950-0dda-4b5f-8601-bc0846be335c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:13.620768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d812950-0dda-4b5f-8601-bc0846be335c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:13.621038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d812950-0dda-4b5f-8601-bc0846be335c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:13.636630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d812950-0dda-4b5f-8601-bc0846be335c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5d812950-0dda-4b5f-8601-bc0846be335c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:13.643156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:13.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:13.647231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 105/201] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:13.651587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d812950-0dda-4b5f-8601-bc0846be335c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:13.660168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:13.713207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b9eb9-d3a2-4bdb-abf5-b3fd60ba8a33 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:13.758288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:13.776274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10a389-d7cc-44a9-8728-3e872f02f80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:13.776902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10a389-d7cc-44a9-8728-3e872f02f80f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:13.777214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10a389-d7cc-44a9-8728-3e872f02f80f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:13.777481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb10a389-d7cc-44a9-8728-3e872f02f80f None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:13.779353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10a389-d7cc-44a9-8728-3e872f02f80f None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:13.780856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb10a389-d7cc-44a9-8728-3e872f02f80f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:13.782533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 97/202] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:13 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:13.788987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10a389-d7cc-44a9-8728-3e872f02f80f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:13.827521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d812950-0dda-4b5f-8601-bc0846be335c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:13.910838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d137b-a912-49dd-aa58-9e609946baab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:13.911805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d137b-a912-49dd-aa58-9e609946baab None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:13.912286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d137b-a912-49dd-aa58-9e609946baab None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:13.912755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984d137b-a912-49dd-aa58-9e609946baab None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:13.913804 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d137b-a912-49dd-aa58-9e609946baab None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:13.914306 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d137b-a912-49dd-aa58-9e609946baab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:13.916660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984d137b-a912-49dd-aa58-9e609946baab None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:13.930348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 106/203] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:13.932879 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb10a389-d7cc-44a9-8728-3e872f02f80f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:13.936415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d137b-a912-49dd-aa58-9e609946baab None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:13.977509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:13.986857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710f98d-5154-4020-aff6-652885afbce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:13.987915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710f98d-5154-4020-aff6-652885afbce6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:13.988397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710f98d-5154-4020-aff6-652885afbce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:13.988934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710f98d-5154-4020-aff6-652885afbce6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:13.990111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710f98d-5154-4020-aff6-652885afbce6 None admin] RBAC: Adding query filter params (role.id=dea086603b04451680a14ae806205d79, scope.project.id=a36c670998e84f8ba575f12eebf55432, user.id=254c256c63404b3fa858d7b68b45a470) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:13.990621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710f98d-5154-4020-aff6-652885afbce6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:13.992279 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1710f98d-5154-4020-aff6-652885afbce6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:13.999880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 98/204] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:13 2022] GET /identity/v3/role_assignments?user.id=254c256c63404b3fa858d7b68b45a470&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=dea086603b04451680a14ae806205d79 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:14.087457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984d137b-a912-49dd-aa58-9e609946baab None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:14.127185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:14.138855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bf3e3-ff61-454e-bcad-73b5cb7b8be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:14.139614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bf3e3-ff61-454e-bcad-73b5cb7b8be4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:14.139882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bf3e3-ff61-454e-bcad-73b5cb7b8be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:14.140255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310bf3e3-ff61-454e-bcad-73b5cb7b8be4 None admin] PATH_INFO: `/v3/users/9c1646303f474283bd07c9c93a1887d8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:14.141545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bf3e3-ff61-454e-bcad-73b5cb7b8be4 None admin] RBAC: Authorizing `identity:get_user(user_id=9c1646303f474283bd07c9c93a1887d8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:14.143028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310bf3e3-ff61-454e-bcad-73b5cb7b8be4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:14.144940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 107/205] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:13 2022] GET /identity/v3/users/9c1646303f474283bd07c9c93a1887d8 => generated 289 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:14.150556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710f98d-5154-4020-aff6-652885afbce6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:14.298441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710f98d-5154-4020-aff6-652885afbce6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:14.334479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0c0e1-8b6f-4be5-b50b-36e94ff8c9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:14.335091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0c0e1-8b6f-4be5-b50b-36e94ff8c9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:14.335406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0c0e1-8b6f-4be5-b50b-36e94ff8c9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:14.335682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0c0e1-8b6f-4be5-b50b-36e94ff8c9e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:14.397065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:14.413010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0c0e1-8b6f-4be5-b50b-36e94ff8c9e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:14.414564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0c0e1-8b6f-4be5-b50b-36e94ff8c9e4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:14.541817 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 99/206] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:14.548352 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bf3e3-ff61-454e-bcad-73b5cb7b8be4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:14.726186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310bf3e3-ff61-454e-bcad-73b5cb7b8be4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:14.776144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4208ec9-51b4-4021-adb1-92c7058ae2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:14.776776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4208ec9-51b4-4021-adb1-92c7058ae2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:14.777062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4208ec9-51b4-4021-adb1-92c7058ae2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:14.777368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4208ec9-51b4-4021-adb1-92c7058ae2f4 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:14.778504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4208ec9-51b4-4021-adb1-92c7058ae2f4 None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:14.780775 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4208ec9-51b4-4021-adb1-92c7058ae2f4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:14.782753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 108/207] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:14 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:14.788271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0c0e1-8b6f-4be5-b50b-36e94ff8c9e4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:14.917005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:14.930293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0c0e1-8b6f-4be5-b50b-36e94ff8c9e4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:14.976558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6440b-d513-4e79-9c1a-399c5ca0a8e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:14.977086 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6440b-d513-4e79-9c1a-399c5ca0a8e0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:14.977387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6440b-d513-4e79-9c1a-399c5ca0a8e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:14.977660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6440b-d513-4e79-9c1a-399c5ca0a8e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:14.978381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6440b-d513-4e79-9c1a-399c5ca0a8e0 None admin] RBAC: Adding query filter params (role.id=1f60cad77a004587a328e9dff0cae088, scope.project.id=8087b41e58fb4aca97cbfced2c011450, user.id=9c1646303f474283bd07c9c93a1887d8) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:14.978685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6440b-d513-4e79-9c1a-399c5ca0a8e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:14.980114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6440b-d513-4e79-9c1a-399c5ca0a8e0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:14.988103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 100/208] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:14 2022] GET /identity/v3/role_assignments?user.id=9c1646303f474283bd07c9c93a1887d8&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=1f60cad77a004587a328e9dff0cae088 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:15.698297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cba004-335a-4321-a6ff-0b8b9e33dae7 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:15.698799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cba004-335a-4321-a6ff-0b8b9e33dae7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:15.698897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cba004-335a-4321-a6ff-0b8b9e33dae7 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:15.700410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 109/209] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:15.707006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723acd5-d1df-4811-b99b-401e89d74081 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:15.707464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723acd5-d1df-4811-b99b-401e89d74081 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:15.707548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723acd5-d1df-4811-b99b-401e89d74081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:15.727018 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6723acd5-d1df-4811-b99b-401e89d74081 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:15.728217 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6723acd5-d1df-4811-b99b-401e89d74081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:15.798512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 101/210] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:16.058437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70285030-cc2e-4813-b24d-508756acf87e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.058980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70285030-cc2e-4813-b24d-508756acf87e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.059461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70285030-cc2e-4813-b24d-508756acf87e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.061575 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 110/211] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:16.067067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7525f89-25bc-4873-ae5a-63492da6ec00 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.067620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7525f89-25bc-4873-ae5a-63492da6ec00 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.068083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7525f89-25bc-4873-ae5a-63492da6ec00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.073718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70285030-cc2e-4813-b24d-508756acf87e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:16.093558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c7525f89-25bc-4873-ae5a-63492da6ec00 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:16.095305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c7525f89-25bc-4873-ae5a-63492da6ec00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:16.149193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:16.170303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 102/212] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:16.172309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04068a2-3998-4c14-b81c-839078eaaf95 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.172754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04068a2-3998-4c14-b81c-839078eaaf95 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.173244 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04068a2-3998-4c14-b81c-839078eaaf95 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.174673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 103/213] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:16.181416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ff3eb-041f-43ec-a2a7-e3dc8b941dea None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.181922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ff3eb-041f-43ec-a2a7-e3dc8b941dea None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.182405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ff3eb-041f-43ec-a2a7-e3dc8b941dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.198160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70285030-cc2e-4813-b24d-508756acf87e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:16.207433 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-008ff3eb-041f-43ec-a2a7-e3dc8b941dea None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:16.209118 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-008ff3eb-041f-43ec-a2a7-e3dc8b941dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:16.233716 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:16.241615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e46f6-2944-4a08-826d-d8e35df5984d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:16.242517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e46f6-2944-4a08-826d-d8e35df5984d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.243044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e46f6-2944-4a08-826d-d8e35df5984d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.243625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e46f6-2944-4a08-826d-d8e35df5984d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.256098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e46f6-2944-4a08-826d-d8e35df5984d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c99e46f6-2944-4a08-826d-d8e35df5984d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:16.261836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:16.262723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:16.263505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:16.270858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 111/214] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:16.272035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e46f6-2944-4a08-826d-d8e35df5984d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:16.285877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 104/215] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:16.382682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e46f6-2944-4a08-826d-d8e35df5984d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:16.409738 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:16.418376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2441c4-a2ee-4610-911b-e972d1902ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:16.419218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2441c4-a2ee-4610-911b-e972d1902ca1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.419812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2441c4-a2ee-4610-911b-e972d1902ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.420304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2441c4-a2ee-4610-911b-e972d1902ca1 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.421810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2441c4-a2ee-4610-911b-e972d1902ca1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:16.422257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2441c4-a2ee-4610-911b-e972d1902ca1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:16.423810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2441c4-a2ee-4610-911b-e972d1902ca1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:16.432607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 112/216] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:16.442207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ff3eb-041f-43ec-a2a7-e3dc8b941dea None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:16.466254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2441c4-a2ee-4610-911b-e972d1902ca1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:16.548751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ff3eb-041f-43ec-a2a7-e3dc8b941dea None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:16.577186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2441c4-a2ee-4610-911b-e972d1902ca1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:16.586125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74050d-2e5e-4a14-b8a7-cd120f1b25b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:16.587034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74050d-2e5e-4a14-b8a7-cd120f1b25b4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.587455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74050d-2e5e-4a14-b8a7-cd120f1b25b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.588014 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db74050d-2e5e-4a14-b8a7-cd120f1b25b4 None admin] PATH_INFO: `/v3/users/6127b7c574304a3d9a61e4caf2922003` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.605034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74050d-2e5e-4a14-b8a7-cd120f1b25b4 None admin] RBAC: Authorizing `identity:get_user(user_id=6127b7c574304a3d9a61e4caf2922003)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:16.606648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db74050d-2e5e-4a14-b8a7-cd120f1b25b4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:16.613849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 105/217] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:16 2022] GET /identity/v3/users/6127b7c574304a3d9a61e4caf2922003 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:16.613990 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ba928-491c-4f8d-af06-9896814eaaea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:16.614650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba928-491c-4f8d-af06-9896814eaaea None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.614826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74050d-2e5e-4a14-b8a7-cd120f1b25b4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:16.614931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba928-491c-4f8d-af06-9896814eaaea None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.615175 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3ba928-491c-4f8d-af06-9896814eaaea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.618443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:16.626281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3ba928-491c-4f8d-af06-9896814eaaea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-8d3ba928-491c-4f8d-af06-9896814eaaea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:16.631487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:16.632573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 113/218] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:56:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:16.638278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ba928-491c-4f8d-af06-9896814eaaea None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:16.688538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:16.723259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db74050d-2e5e-4a14-b8a7-cd120f1b25b4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:16.739124 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3ba928-491c-4f8d-af06-9896814eaaea None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:16.757462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657eebe-0305-468a-9159-6c7923e96204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:16.758087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657eebe-0305-468a-9159-6c7923e96204 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.758365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657eebe-0305-468a-9159-6c7923e96204 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.758637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657eebe-0305-468a-9159-6c7923e96204 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.759736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657eebe-0305-468a-9159-6c7923e96204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:16.761204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657eebe-0305-468a-9159-6c7923e96204 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:16.763464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 106/219] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:16.763596 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657eebe-0305-468a-9159-6c7923e96204 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:16.773166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1acd28-fd9c-4815-b1b8-02ac6bcd0cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:16.773776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1acd28-fd9c-4815-b1b8-02ac6bcd0cfd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.774051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1acd28-fd9c-4815-b1b8-02ac6bcd0cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.774321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1acd28-fd9c-4815-b1b8-02ac6bcd0cfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.834755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:16.844648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1acd28-fd9c-4815-b1b8-02ac6bcd0cfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:16.846143 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1acd28-fd9c-4815-b1b8-02ac6bcd0cfd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:16.865751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657eebe-0305-468a-9159-6c7923e96204 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:16.893821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:16.898875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb763982-fe10-485f-8fa1-fc8547a8585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:16.899476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb763982-fe10-485f-8fa1-fc8547a8585c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.899748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb763982-fe10-485f-8fa1-fc8547a8585c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.900014 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb763982-fe10-485f-8fa1-fc8547a8585c None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.900663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb763982-fe10-485f-8fa1-fc8547a8585c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:16.901228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb763982-fe10-485f-8fa1-fc8547a8585c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:16.902350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb763982-fe10-485f-8fa1-fc8547a8585c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:16.910651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 107/220] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:16.911780 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9682fe-e87f-48e7-9164-c78365a158ac None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:16.912058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9682fe-e87f-48e7-9164-c78365a158ac None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:16.912326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9682fe-e87f-48e7-9164-c78365a158ac None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:16.917155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 108/221] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:16.917155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9682fe-e87f-48e7-9164-c78365a158ac None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:16.948026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 114/222] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:16.948939 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1acd28-fd9c-4815-b1b8-02ac6bcd0cfd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:17.018181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9682fe-e87f-48e7-9164-c78365a158ac None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:17.050487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1acd28-fd9c-4815-b1b8-02ac6bcd0cfd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:17.052525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947473cf-2e1e-4e0f-bd92-201c1bacb03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:17.053437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947473cf-2e1e-4e0f-bd92-201c1bacb03a None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:17.053437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947473cf-2e1e-4e0f-bd92-201c1bacb03a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:17.053672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947473cf-2e1e-4e0f-bd92-201c1bacb03a None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:17.054314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947473cf-2e1e-4e0f-bd92-201c1bacb03a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:17.056101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947473cf-2e1e-4e0f-bd92-201c1bacb03a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:17.064121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:17.078178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 109/223] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:56:16 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:56:17.078178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9905-fad4-45f7-a2ab-2a933555cf3a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:17.078178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9905-fad4-45f7-a2ab-2a933555cf3a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:17.078178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea9905-fad4-45f7-a2ab-2a933555cf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:17.094021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d51901-3409-4ba8-9285-74eb359d74f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:17.094691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d51901-3409-4ba8-9285-74eb359d74f6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:17.095937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d51901-3409-4ba8-9285-74eb359d74f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:17.096212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d51901-3409-4ba8-9285-74eb359d74f6 None admin] PATH_INFO: `/v3/users/254c256c63404b3fa858d7b68b45a470` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:17.097647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d51901-3409-4ba8-9285-74eb359d74f6 None admin] RBAC: Authorizing `identity:get_user(user_id=254c256c63404b3fa858d7b68b45a470)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:17.099162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d51901-3409-4ba8-9285-74eb359d74f6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:17.105764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 115/224] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:16 2022] GET /identity/v3/users/254c256c63404b3fa858d7b68b45a470 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:17.105764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d51901-3409-4ba8-9285-74eb359d74f6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:17.107647 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d4ea9905-fad4-45f7-a2ab-2a933555cf3a None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:17.109604 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d4ea9905-fad4-45f7-a2ab-2a933555cf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:17.127616 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:17.193227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 110/225] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:17.193227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea9905-fad4-45f7-a2ab-2a933555cf3a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:17.242873 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d51901-3409-4ba8-9285-74eb359d74f6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:17.283937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aef18b-c2cf-4b4c-a3aa-2996747965bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:17.284690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef18b-c2cf-4b4c-a3aa-2996747965bd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:17.284899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef18b-c2cf-4b4c-a3aa-2996747965bd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:17.285178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aef18b-c2cf-4b4c-a3aa-2996747965bd None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:17.286440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aef18b-c2cf-4b4c-a3aa-2996747965bd None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:17.287723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aef18b-c2cf-4b4c-a3aa-2996747965bd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:17.289539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 116/226] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:17 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:17.295338 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aef18b-c2cf-4b4c-a3aa-2996747965bd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:17.316075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea9905-fad4-45f7-a2ab-2a933555cf3a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:17.340516 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:17.363327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56d61e-f47d-4242-be25-c3afe8ef28ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:17.363967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56d61e-f47d-4242-be25-c3afe8ef28ee None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:17.364386 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56d61e-f47d-4242-be25-c3afe8ef28ee None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:17.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df56d61e-f47d-4242-be25-c3afe8ef28ee None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:17.365907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56d61e-f47d-4242-be25-c3afe8ef28ee None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:17.367342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df56d61e-f47d-4242-be25-c3afe8ef28ee None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:17.369144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 111/227] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:17 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:17.376723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56d61e-f47d-4242-be25-c3afe8ef28ee None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:17.402348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:17.426521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aef18b-c2cf-4b4c-a3aa-2996747965bd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:17.471427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3a3ae-0037-442f-a724-1742afd9eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:17.471965 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a3ae-0037-442f-a724-1742afd9eca7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:17.472227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a3ae-0037-442f-a724-1742afd9eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:17.472546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3a3ae-0037-442f-a724-1742afd9eca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:17.473353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3a3ae-0037-442f-a724-1742afd9eca7 None admin] RBAC: Adding query filter params (role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, scope.project.id=a36c670998e84f8ba575f12eebf55432, user.id=6127b7c574304a3d9a61e4caf2922003) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:17.473656 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3a3ae-0037-442f-a724-1742afd9eca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:17.475085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3a3ae-0037-442f-a724-1742afd9eca7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:17.481443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 117/228] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:17 2022] GET /identity/v3/role_assignments?user.id=6127b7c574304a3d9a61e4caf2922003&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:17.482254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3a3ae-0037-442f-a724-1742afd9eca7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:17.500898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df56d61e-f47d-4242-be25-c3afe8ef28ee None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:17.548092 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad46fc-45a5-4284-87a1-fd75685db0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:17.548776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad46fc-45a5-4284-87a1-fd75685db0eb None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:17.549050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad46fc-45a5-4284-87a1-fd75685db0eb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:17.549369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dad46fc-45a5-4284-87a1-fd75685db0eb None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432/users/254c256c63404b3fa858d7b68b45a470/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:17.551386 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad46fc-45a5-4284-87a1-fd75685db0eb None admin] RBAC: Authorizing `identity:create_grant(project_id=a36c670998e84f8ba575f12eebf55432, user_id=254c256c63404b3fa858d7b68b45a470, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:17.552862 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dad46fc-45a5-4284-87a1-fd75685db0eb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:17.571122 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 112/229] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:56:17 2022] PUT /identity/v3/projects/a36c670998e84f8ba575f12eebf55432/users/254c256c63404b3fa858d7b68b45a470/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 21:56:17.622306 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3a3ae-0037-442f-a724-1742afd9eca7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:17.664213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f40737-8ff8-4908-9dae-1bb86dffb5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:17.665355 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f40737-8ff8-4908-9dae-1bb86dffb5bb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:17.665872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f40737-8ff8-4908-9dae-1bb86dffb5bb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:17.666370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f40737-8ff8-4908-9dae-1bb86dffb5bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:17.681160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f40737-8ff8-4908-9dae-1bb86dffb5bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-06f40737-8ff8-4908-9dae-1bb86dffb5bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:17.688075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:17.689099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:17.689922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:17.696141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 118/230] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:17.700406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad46fc-45a5-4284-87a1-fd75685db0eb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:17.761497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:17.821904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dad46fc-45a5-4284-87a1-fd75685db0eb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:17.859682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25234c-cd3f-4e1d-9e82-d129e305872c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:17.860527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25234c-cd3f-4e1d-9e82-d129e305872c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:17.860982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25234c-cd3f-4e1d-9e82-d129e305872c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:17.861408 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25234c-cd3f-4e1d-9e82-d129e305872c None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:17.862201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25234c-cd3f-4e1d-9e82-d129e305872c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:17.862638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25234c-cd3f-4e1d-9e82-d129e305872c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:17.864237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25234c-cd3f-4e1d-9e82-d129e305872c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:17.873152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 113/231] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:17.879882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f40737-8ff8-4908-9dae-1bb86dffb5bb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:17.901632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:18.023526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f40737-8ff8-4908-9dae-1bb86dffb5bb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:18.073752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fd8a9-d2e5-4873-822d-0bc8bf95e153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:18.074483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fd8a9-d2e5-4873-822d-0bc8bf95e153 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:18.074766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fd8a9-d2e5-4873-822d-0bc8bf95e153 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:18.075039 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fd8a9-d2e5-4873-822d-0bc8bf95e153 None admin] PATH_INFO: `/v3/users/254c256c63404b3fa858d7b68b45a470` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:18.078066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3fd8a9-d2e5-4873-822d-0bc8bf95e153 None admin] RBAC: Authorizing `identity:get_user(user_id=254c256c63404b3fa858d7b68b45a470)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:18.079604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3fd8a9-d2e5-4873-822d-0bc8bf95e153 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:18.081626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 119/232] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:17 2022] GET /identity/v3/users/254c256c63404b3fa858d7b68b45a470 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:18.086966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25234c-cd3f-4e1d-9e82-d129e305872c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:18.211795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25234c-cd3f-4e1d-9e82-d129e305872c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:18.226578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:18.253774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4650b91-e272-4fe0-91c4-2ae4685861cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:18.254459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4650b91-e272-4fe0-91c4-2ae4685861cf None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:18.254737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4650b91-e272-4fe0-91c4-2ae4685861cf None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:18.255007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4650b91-e272-4fe0-91c4-2ae4685861cf None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:18.256928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4650b91-e272-4fe0-91c4-2ae4685861cf None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:18.258393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4650b91-e272-4fe0-91c4-2ae4685861cf None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:18.260033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 114/233] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:18 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:18.266736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fd8a9-d2e5-4873-822d-0bc8bf95e153 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:18.384650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:18.416775 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fd8a9-d2e5-4873-822d-0bc8bf95e153 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:18.470728 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b251c-7ce6-44e2-aa2f-5a9d388cbeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:18.471401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b251c-7ce6-44e2-aa2f-5a9d388cbeef None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:18.471682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b251c-7ce6-44e2-aa2f-5a9d388cbeef None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:18.471953 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b251c-7ce6-44e2-aa2f-5a9d388cbeef None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432/users/254c256c63404b3fa858d7b68b45a470/roles/dea086603b04451680a14ae806205d79` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:18.474215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b251c-7ce6-44e2-aa2f-5a9d388cbeef None admin] RBAC: Authorizing `identity:create_grant(project_id=a36c670998e84f8ba575f12eebf55432, user_id=254c256c63404b3fa858d7b68b45a470, role_id=dea086603b04451680a14ae806205d79)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:18.475673 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b251c-7ce6-44e2-aa2f-5a9d388cbeef None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:18.493044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 120/234] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:56:18 2022] PUT /identity/v3/projects/a36c670998e84f8ba575f12eebf55432/users/254c256c63404b3fa858d7b68b45a470/roles/dea086603b04451680a14ae806205d79 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:20.114704 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9ea63-ffe0-43e4-8496-5fe4a380b34d None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:20.115212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9ea63-ffe0-43e4-8496-5fe4a380b34d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:20.115286 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a9ea63-ffe0-43e4-8496-5fe4a380b34d None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:20.116824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 115/235] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:20.122240 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022925f5-6660-4956-92ac-9382b0907de5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:20.122650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022925f5-6660-4956-92ac-9382b0907de5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:20.122833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022925f5-6660-4956-92ac-9382b0907de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:20.146551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-022925f5-6660-4956-92ac-9382b0907de5 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:20.147762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-022925f5-6660-4956-92ac-9382b0907de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:20.183712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310fb24-9372-44b7-b112-0e7321d9ed31 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:20.184026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310fb24-9372-44b7-b112-0e7321d9ed31 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:20.184253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310fb24-9372-44b7-b112-0e7321d9ed31 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:20.186032 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 116/236] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:20.199289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a73809-a7fa-427a-8129-4b5577bb08c9 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:20.199578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a73809-a7fa-427a-8129-4b5577bb08c9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:20.199857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a73809-a7fa-427a-8129-4b5577bb08c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:20.231889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c5a73809-a7fa-427a-8129-4b5577bb08c9 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:20.234426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c5a73809-a7fa-427a-8129-4b5577bb08c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:20.237513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 121/237] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:20.238783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea33bc-1676-48fb-9368-e2d57a5ba47e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:20.239101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea33bc-1676-48fb-9368-e2d57a5ba47e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:20.239319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea33bc-1676-48fb-9368-e2d57a5ba47e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:20.248822 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 122/238] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:20 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:20.256583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67870bb0-676a-4f8f-a9b8-010b7c55fedd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:20.256866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67870bb0-676a-4f8f-a9b8-010b7c55fedd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:20.257162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67870bb0-676a-4f8f-a9b8-010b7c55fedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:20.290945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-67870bb0-676a-4f8f-a9b8-010b7c55fedd None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:20.292142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-67870bb0-676a-4f8f-a9b8-010b7c55fedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:20.360271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 117/239] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:20.389593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:20.397494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 123/240] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:20.504141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e9777-763e-4c09-b460-38cb69620f4d None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:20.504456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e9777-763e-4c09-b460-38cb69620f4d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:20.504696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e9777-763e-4c09-b460-38cb69620f4d None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:20.506218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 118/241] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:20.511610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9bee1-0946-4fe0-a21e-9b767334df9b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:20.512126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9bee1-0946-4fe0-a21e-9b767334df9b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:20.512585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9bee1-0946-4fe0-a21e-9b767334df9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:20.541278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-80c9bee1-0946-4fe0-a21e-9b767334df9b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:20.542757 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-80c9bee1-0946-4fe0-a21e-9b767334df9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:20.601519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e9777-763e-4c09-b460-38cb69620f4d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:20.646941 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 124/242] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:20.689511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9bee1-0946-4fe0-a21e-9b767334df9b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:20.744448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e9777-763e-4c09-b460-38cb69620f4d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:20.758313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:20.803987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e15a28-8694-482e-820b-0bf855173e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:20.804613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e15a28-8694-482e-820b-0bf855173e7d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:20.804884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e15a28-8694-482e-820b-0bf855173e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:20.805190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e15a28-8694-482e-820b-0bf855173e7d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:20.816716 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e15a28-8694-482e-820b-0bf855173e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f5e15a28-8694-482e-820b-0bf855173e7d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:20.821753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:20.822609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:20.823393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 119/243] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:20.824033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e15a28-8694-482e-820b-0bf855173e7d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:20.842903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9bee1-0946-4fe0-a21e-9b767334df9b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:20.873955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:20.882943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fbae6-5201-4aa0-a9c8-74cf113bdb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:20.883786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fbae6-5201-4aa0-a9c8-74cf113bdb56 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:20.884131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fbae6-5201-4aa0-a9c8-74cf113bdb56 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:20.884582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48fbae6-5201-4aa0-a9c8-74cf113bdb56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:20.896108 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48fbae6-5201-4aa0-a9c8-74cf113bdb56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c48fbae6-5201-4aa0-a9c8-74cf113bdb56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:20.901313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:20.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:20.902964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:20.907440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 125/244] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:20.908397 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fbae6-5201-4aa0-a9c8-74cf113bdb56 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:20.941549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e15a28-8694-482e-820b-0bf855173e7d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:20.978024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086b8f5-ca64-45a5-8110-eb309441e21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:20.979095 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b8f5-ca64-45a5-8110-eb309441e21c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:20.979370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b8f5-ca64-45a5-8110-eb309441e21c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:20.979640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5086b8f5-ca64-45a5-8110-eb309441e21c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:20.980799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086b8f5-ca64-45a5-8110-eb309441e21c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:20.982936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5086b8f5-ca64-45a5-8110-eb309441e21c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:20.984745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 120/245] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:20.989905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086b8f5-ca64-45a5-8110-eb309441e21c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:21.019514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48fbae6-5201-4aa0-a9c8-74cf113bdb56 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:21.033003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:21.057543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1901b-6030-428e-9185-32851d68b6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:21.058265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1901b-6030-428e-9185-32851d68b6b2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:21.058543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1901b-6030-428e-9185-32851d68b6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:21.058818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1901b-6030-428e-9185-32851d68b6b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:21.059481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1901b-6030-428e-9185-32851d68b6b2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:21.059783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1901b-6030-428e-9185-32851d68b6b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:21.061585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1901b-6030-428e-9185-32851d68b6b2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:21.074706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 126/246] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:21.076840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1901b-6030-428e-9185-32851d68b6b2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:21.111989 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5086b8f5-ca64-45a5-8110-eb309441e21c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:21.137532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:21.149628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16de67c-ece6-4c7c-8423-66e6934dbca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:21.150244 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16de67c-ece6-4c7c-8423-66e6934dbca5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:21.150529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16de67c-ece6-4c7c-8423-66e6934dbca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:21.150807 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16de67c-ece6-4c7c-8423-66e6934dbca5 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:21.151528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16de67c-ece6-4c7c-8423-66e6934dbca5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:21.151774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16de67c-ece6-4c7c-8423-66e6934dbca5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:21.153396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16de67c-ece6-4c7c-8423-66e6934dbca5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:21.165150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 121/247] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:21.165150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16de67c-ece6-4c7c-8423-66e6934dbca5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:21.198617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1901b-6030-428e-9185-32851d68b6b2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:21.234390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48814ba2-a6ae-4e4e-a130-ec00ddb1d39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:21.235021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48814ba2-a6ae-4e4e-a130-ec00ddb1d39b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:21.235296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48814ba2-a6ae-4e4e-a130-ec00ddb1d39b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:21.235569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48814ba2-a6ae-4e4e-a130-ec00ddb1d39b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:21.245890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48814ba2-a6ae-4e4e-a130-ec00ddb1d39b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-48814ba2-a6ae-4e4e-a130-ec00ddb1d39b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:21.251301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:21.252299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:21.253097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 127/248] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:56:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:21.253765 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48814ba2-a6ae-4e4e-a130-ec00ddb1d39b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:21.265002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:21.268795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16de67c-ece6-4c7c-8423-66e6934dbca5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:21.301312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5a7ba-9ee8-43dd-b6bb-5d783515bad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:21.301881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5a7ba-9ee8-43dd-b6bb-5d783515bad8 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:21.302156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5a7ba-9ee8-43dd-b6bb-5d783515bad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:21.302431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5a7ba-9ee8-43dd-b6bb-5d783515bad8 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:21.303071 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5a7ba-9ee8-43dd-b6bb-5d783515bad8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:21.304498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5a7ba-9ee8-43dd-b6bb-5d783515bad8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:21.325032 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 122/249] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:56:21 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:56:21.330075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5a7ba-9ee8-43dd-b6bb-5d783515bad8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:21.364097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48814ba2-a6ae-4e4e-a130-ec00ddb1d39b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:21.377071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:21.406773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee551224-ccb3-442c-b263-eb36a152845a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:21.407333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee551224-ccb3-442c-b263-eb36a152845a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:21.407611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee551224-ccb3-442c-b263-eb36a152845a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:21.407884 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee551224-ccb3-442c-b263-eb36a152845a None admin] PATH_INFO: `/v3/users/bcee039de8d54d1f82ec9c49f7b5e692` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:21.423433 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee551224-ccb3-442c-b263-eb36a152845a None admin] RBAC: Authorizing `identity:get_user(user_id=bcee039de8d54d1f82ec9c49f7b5e692)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:21.424946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee551224-ccb3-442c-b263-eb36a152845a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:21.426742 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 128/250] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:21 2022] GET /identity/v3/users/bcee039de8d54d1f82ec9c49f7b5e692 => generated 297 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:21.428038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f6a86-03c9-477b-8e5f-ab90acdaefa5 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:21.428320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f6a86-03c9-477b-8e5f-ab90acdaefa5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:21.428592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0f6a86-03c9-477b-8e5f-ab90acdaefa5 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:21.433234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 129/251] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:21.433234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f6a86-03c9-477b-8e5f-ab90acdaefa5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:21.470931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5a7ba-9ee8-43dd-b6bb-5d783515bad8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:21.509830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfb360-23f2-493f-82c8-4a0bd9ecf10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:21.510470 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfb360-23f2-493f-82c8-4a0bd9ecf10c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:21.510742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfb360-23f2-493f-82c8-4a0bd9ecf10c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:21.511018 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabfb360-23f2-493f-82c8-4a0bd9ecf10c None admin] PATH_INFO: `/v3/users/6127b7c574304a3d9a61e4caf2922003` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:21.512754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfb360-23f2-493f-82c8-4a0bd9ecf10c None admin] RBAC: Authorizing `identity:get_user(user_id=6127b7c574304a3d9a61e4caf2922003)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:21.515155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabfb360-23f2-493f-82c8-4a0bd9ecf10c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:21.517297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 123/252] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:21 2022] GET /identity/v3/users/6127b7c574304a3d9a61e4caf2922003 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:21.518351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfb360-23f2-493f-82c8-4a0bd9ecf10c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:21.548040 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:21.561588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0f6a86-03c9-477b-8e5f-ab90acdaefa5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:21.607365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860cd47d-ab7c-465d-980b-788ced9491bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:21.608234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cd47d-ab7c-465d-980b-788ced9491bc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:21.608515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cd47d-ab7c-465d-980b-788ced9491bc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:21.609310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860cd47d-ab7c-465d-980b-788ced9491bc None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:21.609994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cd47d-ab7c-465d-980b-788ced9491bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:21.610299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cd47d-ab7c-465d-980b-788ced9491bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:21.612828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860cd47d-ab7c-465d-980b-788ced9491bc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:21.624820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 130/253] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:21.624820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e90c10-6686-40e1-995b-8e13f12586f8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:21.624820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e90c10-6686-40e1-995b-8e13f12586f8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:21.624820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e90c10-6686-40e1-995b-8e13f12586f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:21.643471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabfb360-23f2-493f-82c8-4a0bd9ecf10c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:21.651033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-76e90c10-6686-40e1-995b-8e13f12586f8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:21.652230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-76e90c10-6686-40e1-995b-8e13f12586f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:21.680616 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:21.683460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3864559-df3f-4825-ba4a-b47617b65069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:21.684071 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3864559-df3f-4825-ba4a-b47617b65069 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:21.684346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3864559-df3f-4825-ba4a-b47617b65069 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:21.684616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3864559-df3f-4825-ba4a-b47617b65069 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:21.734481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 131/254] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:21.735183 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e90c10-6686-40e1-995b-8e13f12586f8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:21.807350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3864559-df3f-4825-ba4a-b47617b65069 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:21.808800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3864559-df3f-4825-ba4a-b47617b65069 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:21.857498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:21.896075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e90c10-6686-40e1-995b-8e13f12586f8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:21.931899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283066f-69cd-476f-b1f4-3a691cfd83b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:21.932444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283066f-69cd-476f-b1f4-3a691cfd83b7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:21.932717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283066f-69cd-476f-b1f4-3a691cfd83b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:21.933029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283066f-69cd-476f-b1f4-3a691cfd83b7 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:21.934227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283066f-69cd-476f-b1f4-3a691cfd83b7 None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:21.935705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d283066f-69cd-476f-b1f4-3a691cfd83b7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:21.937670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 132/255] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:21 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:21.938443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283066f-69cd-476f-b1f4-3a691cfd83b7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:21.973454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:22.008389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 124/256] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:22.009235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3864559-df3f-4825-ba4a-b47617b65069 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:22.093523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d283066f-69cd-476f-b1f4-3a691cfd83b7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:22.129633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd089-6b2f-478a-b394-f77992a79702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:22.130298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd089-6b2f-478a-b394-f77992a79702 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:22.130511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd089-6b2f-478a-b394-f77992a79702 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:22.130776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6edd089-6b2f-478a-b394-f77992a79702 None admin] PATH_INFO: `/v3/users/254c256c63404b3fa858d7b68b45a470` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:22.131888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edd089-6b2f-478a-b394-f77992a79702 None admin] RBAC: Authorizing `identity:get_user(user_id=254c256c63404b3fa858d7b68b45a470)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:22.133367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6edd089-6b2f-478a-b394-f77992a79702 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:22.134935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 133/257] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:21 2022] GET /identity/v3/users/254c256c63404b3fa858d7b68b45a470 => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:22.135787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd089-6b2f-478a-b394-f77992a79702 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:22.165812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3864559-df3f-4825-ba4a-b47617b65069 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:22.199080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:22.218341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b3e8-020f-4e86-a5d8-b81271c5a3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:22.219271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b3e8-020f-4e86-a5d8-b81271c5a3e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:22.219694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b3e8-020f-4e86-a5d8-b81271c5a3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:22.220110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f562b3e8-020f-4e86-a5d8-b81271c5a3e4 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432/users/6127b7c574304a3d9a61e4caf2922003/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:22.222395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562b3e8-020f-4e86-a5d8-b81271c5a3e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a36c670998e84f8ba575f12eebf55432, user_id=6127b7c574304a3d9a61e4caf2922003, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:22.224585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f562b3e8-020f-4e86-a5d8-b81271c5a3e4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:22.242756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 125/258] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:56:22 2022] PUT /identity/v3/projects/a36c670998e84f8ba575f12eebf55432/users/6127b7c574304a3d9a61e4caf2922003/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:22.244608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b3e8-020f-4e86-a5d8-b81271c5a3e4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:22.255083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:22.269047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6edd089-6b2f-478a-b394-f77992a79702 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:22.356097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc0180-d51a-4ed8-8f6d-c6db6f98d873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:22.356595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc0180-d51a-4ed8-8f6d-c6db6f98d873 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:22.356869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc0180-d51a-4ed8-8f6d-c6db6f98d873 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:22.357153 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fc0180-d51a-4ed8-8f6d-c6db6f98d873 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:22.359007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc0180-d51a-4ed8-8f6d-c6db6f98d873 None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:22.360483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fc0180-d51a-4ed8-8f6d-c6db6f98d873 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:22.370676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 134/259] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:22 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:22.371752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc0180-d51a-4ed8-8f6d-c6db6f98d873 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:22.407906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f562b3e8-020f-4e86-a5d8-b81271c5a3e4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:22.476755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c01a2-3bc6-4da8-9b13-71eb5130c764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:22.477861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c01a2-3bc6-4da8-9b13-71eb5130c764 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:22.478340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c01a2-3bc6-4da8-9b13-71eb5130c764 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:22.478840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c01a2-3bc6-4da8-9b13-71eb5130c764 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:22.492299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c01a2-3bc6-4da8-9b13-71eb5130c764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-297c01a2-3bc6-4da8-9b13-71eb5130c764 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:22.499354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:22.500319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:22.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:22.504082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 126/260] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:22.509420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c01a2-3bc6-4da8-9b13-71eb5130c764 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:22.522385 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fc0180-d51a-4ed8-8f6d-c6db6f98d873 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:22.577819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:22.592956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad62b0-5f09-4411-96ab-f7cd829bdbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:22.594996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad62b0-5f09-4411-96ab-f7cd829bdbbb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:22.595499 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad62b0-5f09-4411-96ab-f7cd829bdbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:22.595964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad62b0-5f09-4411-96ab-f7cd829bdbbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:22.635470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:22.680073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad62b0-5f09-4411-96ab-f7cd829bdbbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:22.682270 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad62b0-5f09-4411-96ab-f7cd829bdbbb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:22.713049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c01a2-3bc6-4da8-9b13-71eb5130c764 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:22.801620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 135/261] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:22.802981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad62b0-5f09-4411-96ab-f7cd829bdbbb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:22.806395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e5b97-5b01-456f-8c01-7125b91f0f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:22.807702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e5b97-5b01-456f-8c01-7125b91f0f16 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:22.807702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e5b97-5b01-456f-8c01-7125b91f0f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:22.807702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604e5b97-5b01-456f-8c01-7125b91f0f16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:22.808234 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e5b97-5b01-456f-8c01-7125b91f0f16 None admin] RBAC: Adding query filter params (role.id=1f60cad77a004587a328e9dff0cae088, scope.project.id=a36c670998e84f8ba575f12eebf55432, user.id=bcee039de8d54d1f82ec9c49f7b5e692) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:22.808538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e5b97-5b01-456f-8c01-7125b91f0f16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:22.810002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604e5b97-5b01-456f-8c01-7125b91f0f16 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:22.830104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 127/262] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:22 2022] GET /identity/v3/role_assignments?user.id=bcee039de8d54d1f82ec9c49f7b5e692&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=1f60cad77a004587a328e9dff0cae088 => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:22.830860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e5b97-5b01-456f-8c01-7125b91f0f16 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:22.880288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:22.959791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad62b0-5f09-4411-96ab-f7cd829bdbbb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:23.020245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb774383-2ca9-4f29-a930-885693926dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:23.021277 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb774383-2ca9-4f29-a930-885693926dcc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:23.021753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb774383-2ca9-4f29-a930-885693926dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:23.022828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb774383-2ca9-4f29-a930-885693926dcc None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:23.024019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb774383-2ca9-4f29-a930-885693926dcc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:23.024517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb774383-2ca9-4f29-a930-885693926dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:23.026351 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb774383-2ca9-4f29-a930-885693926dcc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:23.044538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 136/263] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:23.062272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb774383-2ca9-4f29-a930-885693926dcc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:23.119142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604e5b97-5b01-456f-8c01-7125b91f0f16 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:23.129436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:23.178111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035a761-7f88-42f7-96c6-8ae126858160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:23.178739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035a761-7f88-42f7-96c6-8ae126858160 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:23.179010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035a761-7f88-42f7-96c6-8ae126858160 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:23.179275 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d035a761-7f88-42f7-96c6-8ae126858160 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:23.180566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035a761-7f88-42f7-96c6-8ae126858160 None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:23.182119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d035a761-7f88-42f7-96c6-8ae126858160 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:23.183746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 128/264] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:22 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:23.190121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035a761-7f88-42f7-96c6-8ae126858160 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:23.275206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb774383-2ca9-4f29-a930-885693926dcc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:23.329595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:23.350684 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1fd14-b849-45fc-affb-b5f62abc4b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:23.351602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1fd14-b849-45fc-affb-b5f62abc4b00 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:23.352056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1fd14-b849-45fc-affb-b5f62abc4b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:23.352575 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1fd14-b849-45fc-affb-b5f62abc4b00 None admin] PATH_INFO: `/v3/users/254c256c63404b3fa858d7b68b45a470` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:23.357048 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1fd14-b849-45fc-affb-b5f62abc4b00 None admin] RBAC: Authorizing `identity:get_user(user_id=254c256c63404b3fa858d7b68b45a470)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:23.359363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1fd14-b849-45fc-affb-b5f62abc4b00 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:23.361326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 137/265] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:23 2022] GET /identity/v3/users/254c256c63404b3fa858d7b68b45a470 => generated 283 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:23.366821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1fd14-b849-45fc-affb-b5f62abc4b00 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:23.406835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d035a761-7f88-42f7-96c6-8ae126858160 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:23.453138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab06d42-64c1-4925-b8e1-dd9e44ad3da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:23.453138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab06d42-64c1-4925-b8e1-dd9e44ad3da1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:23.453138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab06d42-64c1-4925-b8e1-dd9e44ad3da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:23.453138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab06d42-64c1-4925-b8e1-dd9e44ad3da1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:23.453539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab06d42-64c1-4925-b8e1-dd9e44ad3da1 None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=a36c670998e84f8ba575f12eebf55432, user.id=254c256c63404b3fa858d7b68b45a470) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:23.453539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab06d42-64c1-4925-b8e1-dd9e44ad3da1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:23.453539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab06d42-64c1-4925-b8e1-dd9e44ad3da1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:23.461031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 129/266] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:23 2022] GET /identity/v3/role_assignments?user.id=254c256c63404b3fa858d7b68b45a470&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:23.562280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1fd14-b849-45fc-affb-b5f62abc4b00 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:23.608209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddb706-4c28-4ef5-bb90-7913050163a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:23.608810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddb706-4c28-4ef5-bb90-7913050163a5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:23.609088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddb706-4c28-4ef5-bb90-7913050163a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:23.609386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddb706-4c28-4ef5-bb90-7913050163a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:23.696010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:23.706572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ddb706-4c28-4ef5-bb90-7913050163a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:23.708064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ddb706-4c28-4ef5-bb90-7913050163a5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:23.835089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 138/267] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:23.840531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab06d42-64c1-4925-b8e1-dd9e44ad3da1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:23.903133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:23.985997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab06d42-64c1-4925-b8e1-dd9e44ad3da1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:24.027008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d49fc7-4c74-4f4c-a55d-fcca86a4c112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:24.027652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d49fc7-4c74-4f4c-a55d-fcca86a4c112 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:24.027924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d49fc7-4c74-4f4c-a55d-fcca86a4c112 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:24.028191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d49fc7-4c74-4f4c-a55d-fcca86a4c112 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:24.030182 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d49fc7-4c74-4f4c-a55d-fcca86a4c112 None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:24.031635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d49fc7-4c74-4f4c-a55d-fcca86a4c112 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:24.033744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 130/268] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:23 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:24.039223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddb706-4c28-4ef5-bb90-7913050163a5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:24.147282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cdb8c-073c-4071-93d6-d9f4caa45bf0 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:24.147928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cdb8c-073c-4071-93d6-d9f4caa45bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:24.148394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cdb8c-073c-4071-93d6-d9f4caa45bf0 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:24.150207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 131/269] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:24.157633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f1377-b3c3-4890-978b-25974ecee8bf None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:24.157877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f1377-b3c3-4890-978b-25974ecee8bf None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:24.158094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470f1377-b3c3-4890-978b-25974ecee8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:24.173616 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:24.184551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddb706-4c28-4ef5-bb90-7913050163a5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:24.194340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-470f1377-b3c3-4890-978b-25974ecee8bf None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:24.195622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-470f1377-b3c3-4890-978b-25974ecee8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:24.248832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d50b81-38a9-47ce-ba83-c8628ceb2c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:24.249481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d50b81-38a9-47ce-ba83-c8628ceb2c80 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:24.249757 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d50b81-38a9-47ce-ba83-c8628ceb2c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:24.250027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d50b81-38a9-47ce-ba83-c8628ceb2c80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:24.250749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d50b81-38a9-47ce-ba83-c8628ceb2c80 None admin] RBAC: Adding query filter params (role.id=dea086603b04451680a14ae806205d79, scope.project.id=a36c670998e84f8ba575f12eebf55432, user.id=254c256c63404b3fa858d7b68b45a470) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:24.251053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d50b81-38a9-47ce-ba83-c8628ceb2c80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:24.252481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d50b81-38a9-47ce-ba83-c8628ceb2c80 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:24.267023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 139/270] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:24 2022] GET /identity/v3/role_assignments?user.id=254c256c63404b3fa858d7b68b45a470&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=dea086603b04451680a14ae806205d79 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:24.307673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 132/271] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:24.321979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665891a5-ed7a-49df-b6d3-bd8aa5ff57c0 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:24.322527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665891a5-ed7a-49df-b6d3-bd8aa5ff57c0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:24.322527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665891a5-ed7a-49df-b6d3-bd8aa5ff57c0 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:24.323888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 140/272] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:24.339038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559ccea-4bc2-4e07-8c79-63c780035aab None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:24.339339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559ccea-4bc2-4e07-8c79-63c780035aab None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:24.339579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559ccea-4bc2-4e07-8c79-63c780035aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:24.366310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d559ccea-4bc2-4e07-8c79-63c780035aab None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:24.367524 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d559ccea-4bc2-4e07-8c79-63c780035aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:24.406367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:24.470403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 133/273] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:24.601577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665891a5-ed7a-49df-b6d3-bd8aa5ff57c0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:24.721882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665891a5-ed7a-49df-b6d3-bd8aa5ff57c0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:24.761956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb88f92-6605-4e13-99bc-3117150c4d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:24.762589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb88f92-6605-4e13-99bc-3117150c4d82 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:24.762864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb88f92-6605-4e13-99bc-3117150c4d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:24.763130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb88f92-6605-4e13-99bc-3117150c4d82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:24.777668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb88f92-6605-4e13-99bc-3117150c4d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cdb88f92-6605-4e13-99bc-3117150c4d82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:24.786086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:24.786983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:24.787777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:24.788372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 141/274] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:24.795537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559ccea-4bc2-4e07-8c79-63c780035aab None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:24.812545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb88f92-6605-4e13-99bc-3117150c4d82 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:24.846204 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:24.943122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb88f92-6605-4e13-99bc-3117150c4d82 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:24.944777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559ccea-4bc2-4e07-8c79-63c780035aab None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:24.986414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a15557-bba5-482c-8a97-70f43ce1b5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:24.987529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a15557-bba5-482c-8a97-70f43ce1b5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:24.988008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a15557-bba5-482c-8a97-70f43ce1b5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:24.988478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a15557-bba5-482c-8a97-70f43ce1b5b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:24.988801 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5891ad4-3a35-454c-b340-a48f05a45886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:24.989821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5891ad4-3a35-454c-b340-a48f05a45886 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:24.990101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5891ad4-3a35-454c-b340-a48f05a45886 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:24.990370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5891ad4-3a35-454c-b340-a48f05a45886 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:24.990912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a15557-bba5-482c-8a97-70f43ce1b5b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:24.991082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5891ad4-3a35-454c-b340-a48f05a45886 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:24.991343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5891ad4-3a35-454c-b340-a48f05a45886 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:24.992761 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5891ad4-3a35-454c-b340-a48f05a45886 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:24.993231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a15557-bba5-482c-8a97-70f43ce1b5b8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:24.996303 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 142/275] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:25.002379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a15557-bba5-482c-8a97-70f43ce1b5b8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:25.007332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 134/276] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:25.013097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5891ad4-3a35-454c-b340-a48f05a45886 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:25.047053 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:25.098630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:25.114436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a15557-bba5-482c-8a97-70f43ce1b5b8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:25.133496 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5891ad4-3a35-454c-b340-a48f05a45886 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:25.151964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27044385-c4cd-4be9-8c85-cc18bb619cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:25.152574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27044385-c4cd-4be9-8c85-cc18bb619cc8 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:25.153034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27044385-c4cd-4be9-8c85-cc18bb619cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:25.153352 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27044385-c4cd-4be9-8c85-cc18bb619cc8 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:25.154440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27044385-c4cd-4be9-8c85-cc18bb619cc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:25.156287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27044385-c4cd-4be9-8c85-cc18bb619cc8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:25.177924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6241ec-a732-4258-9689-c330bbf645d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:25.178756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6241ec-a732-4258-9689-c330bbf645d3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:25.179302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6241ec-a732-4258-9689-c330bbf645d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:25.179742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6241ec-a732-4258-9689-c330bbf645d3 None admin] PATH_INFO: `/v3/users/18273b3dc79a4c65bff906bf80f41a8b` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:25.186202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 143/277] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:56:25 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:56:25.188295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d1d75-4916-4a5f-a2e2-0d9f9618849b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:25.188871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d1d75-4916-4a5f-a2e2-0d9f9618849b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:25.189445 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3d1d75-4916-4a5f-a2e2-0d9f9618849b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:25.191080 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 144/278] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:25.198096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595d0dc-cd26-45bb-b7f2-10517366664b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:25.198903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595d0dc-cd26-45bb-b7f2-10517366664b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:25.199333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595d0dc-cd26-45bb-b7f2-10517366664b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:25.205552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6241ec-a732-4258-9689-c330bbf645d3 None admin] RBAC: Authorizing `identity:get_user(user_id=18273b3dc79a4c65bff906bf80f41a8b)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:25.207893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6241ec-a732-4258-9689-c330bbf645d3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:25.211287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 135/279] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:25 2022] GET /identity/v3/users/18273b3dc79a4c65bff906bf80f41a8b => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:25.224707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1595d0dc-cd26-45bb-b7f2-10517366664b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:25.226059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1595d0dc-cd26-45bb-b7f2-10517366664b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:25.226161 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6241ec-a732-4258-9689-c330bbf645d3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:25.316532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 145/280] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:25.354158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:25.368849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6241ec-a732-4258-9689-c330bbf645d3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:25.406680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0575b-1ba7-4ecc-86fb-2add83ee4fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:25.407286 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0575b-1ba7-4ecc-86fb-2add83ee4fdb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:25.407562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0575b-1ba7-4ecc-86fb-2add83ee4fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:25.407830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0575b-1ba7-4ecc-86fb-2add83ee4fdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:25.408544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0575b-1ba7-4ecc-86fb-2add83ee4fdb None admin] RBAC: Adding query filter params (role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, user.id=18273b3dc79a4c65bff906bf80f41a8b, scope.system=all) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:25.408850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0575b-1ba7-4ecc-86fb-2add83ee4fdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:25.410385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0575b-1ba7-4ecc-86fb-2add83ee4fdb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:25.416799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 136/281] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 22 21:56:25 2022] GET /identity/v3/role_assignments?user.id=18273b3dc79a4c65bff906bf80f41a8b&scope.system=all&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:25.617511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595d0dc-cd26-45bb-b7f2-10517366664b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:25.693884 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:25.735597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595d0dc-cd26-45bb-b7f2-10517366664b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:25.749451 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9341390-2dbd-4af0-acf5-06cc84da5f92 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:25.749744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9341390-2dbd-4af0-acf5-06cc84da5f92 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:25.749998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9341390-2dbd-4af0-acf5-06cc84da5f92 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:25.755426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 137/282] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:25.756638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb86dd-207c-4c69-89fc-5df78153a091 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:25.756919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb86dd-207c-4c69-89fc-5df78153a091 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:25.757213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb86dd-207c-4c69-89fc-5df78153a091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:25.784369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3259375-970a-404c-8900-a251b97bac74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:25.785016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3259375-970a-404c-8900-a251b97bac74 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:25.785316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3259375-970a-404c-8900-a251b97bac74 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:25.785580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3259375-970a-404c-8900-a251b97bac74 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:25.786973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-bbfb86dd-207c-4c69-89fc-5df78153a091 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:25.788154 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bbfb86dd-207c-4c69-89fc-5df78153a091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:25.816442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3259375-970a-404c-8900-a251b97bac74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e3259375-970a-404c-8900-a251b97bac74 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:25.821297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:25.822158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:25.822954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:25.830463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 146/283] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:25.835981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3259375-970a-404c-8900-a251b97bac74 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:25.931835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 138/284] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:25.963889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3259375-970a-404c-8900-a251b97bac74 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:25.995082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:26.009827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ed350-a6a0-49a5-813f-d5c03ed7afee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:26.010452 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ed350-a6a0-49a5-813f-d5c03ed7afee None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:26.010735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ed350-a6a0-49a5-813f-d5c03ed7afee None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:26.011011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4ed350-a6a0-49a5-813f-d5c03ed7afee None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:26.011751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ed350-a6a0-49a5-813f-d5c03ed7afee None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:26.012061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ed350-a6a0-49a5-813f-d5c03ed7afee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:26.014027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4ed350-a6a0-49a5-813f-d5c03ed7afee None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:26.023247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 147/285] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:26.028860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb86dd-207c-4c69-89fc-5df78153a091 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:26.138956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:26.162993 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb86dd-207c-4c69-89fc-5df78153a091 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:26.221911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46fcc0-3226-40a7-bfa4-77e6bc74cefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:26.222512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46fcc0-3226-40a7-bfa4-77e6bc74cefe None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:26.222878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46fcc0-3226-40a7-bfa4-77e6bc74cefe None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:26.223143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46fcc0-3226-40a7-bfa4-77e6bc74cefe None admin] PATH_INFO: `/v3/users/6127b7c574304a3d9a61e4caf2922003` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:26.224347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46fcc0-3226-40a7-bfa4-77e6bc74cefe None admin] RBAC: Authorizing `identity:get_user(user_id=6127b7c574304a3d9a61e4caf2922003)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:26.225797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed46fcc0-3226-40a7-bfa4-77e6bc74cefe None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:26.227373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 139/286] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:26 2022] GET /identity/v3/users/6127b7c574304a3d9a61e4caf2922003 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:26.233526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ed350-a6a0-49a5-813f-d5c03ed7afee None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:26.238876 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46fcc0-3226-40a7-bfa4-77e6bc74cefe None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:26.351639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46fcc0-3226-40a7-bfa4-77e6bc74cefe None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:26.395868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced9213-3387-44c0-bd7a-7ddadb5af1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:26.396495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced9213-3387-44c0-bd7a-7ddadb5af1bc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:26.396770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced9213-3387-44c0-bd7a-7ddadb5af1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:26.397034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ced9213-3387-44c0-bd7a-7ddadb5af1bc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:26.422280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ced9213-3387-44c0-bd7a-7ddadb5af1bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:26.430157 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4ed350-a6a0-49a5-813f-d5c03ed7afee None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-8ced9213-3387-44c0-bd7a-7ddadb5af1bc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:26.434004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:26.434886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:26.435926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 140/287] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:26.441316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced9213-3387-44c0-bd7a-7ddadb5af1bc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:26.471118 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:26.507721 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b013f5-dd75-492c-8e50-e372248e04a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:26.508773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b013f5-dd75-492c-8e50-e372248e04a9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:26.509052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b013f5-dd75-492c-8e50-e372248e04a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:26.509398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b013f5-dd75-492c-8e50-e372248e04a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:26.576727 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:26.585042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b013f5-dd75-492c-8e50-e372248e04a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:26.586799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b013f5-dd75-492c-8e50-e372248e04a9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:26.660830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ced9213-3387-44c0-bd7a-7ddadb5af1bc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:26.702208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280b18e-112f-4e5a-b208-e390bb7508ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:26.702886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280b18e-112f-4e5a-b208-e390bb7508ee None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:26.703172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280b18e-112f-4e5a-b208-e390bb7508ee None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:26.703519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c280b18e-112f-4e5a-b208-e390bb7508ee None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:26.704183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280b18e-112f-4e5a-b208-e390bb7508ee None admin] RBAC: Adding query filter params (name=reader) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:26.704559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280b18e-112f-4e5a-b208-e390bb7508ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:26.706042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c280b18e-112f-4e5a-b208-e390bb7508ee None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:26.714733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 141/288] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:26.725409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280b18e-112f-4e5a-b208-e390bb7508ee None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:26.787325 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 148/289] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:26.795448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b013f5-dd75-492c-8e50-e372248e04a9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:26.813523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:26.849070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c280b18e-112f-4e5a-b208-e390bb7508ee None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:26.889688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585749a4-c569-42dc-aa75-1ec1c51f402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:26.890370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585749a4-c569-42dc-aa75-1ec1c51f402b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:26.890653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585749a4-c569-42dc-aa75-1ec1c51f402b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:26.890927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585749a4-c569-42dc-aa75-1ec1c51f402b None admin] PATH_INFO: `/v3/users/bcee039de8d54d1f82ec9c49f7b5e692` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:26.892096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585749a4-c569-42dc-aa75-1ec1c51f402b None admin] RBAC: Authorizing `identity:get_user(user_id=bcee039de8d54d1f82ec9c49f7b5e692)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:26.893572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585749a4-c569-42dc-aa75-1ec1c51f402b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:26.896898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 142/290] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:26 2022] GET /identity/v3/users/bcee039de8d54d1f82ec9c49f7b5e692 => generated 297 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:26.904552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585749a4-c569-42dc-aa75-1ec1c51f402b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:27.026841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b013f5-dd75-492c-8e50-e372248e04a9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:27.028230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585749a4-c569-42dc-aa75-1ec1c51f402b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:27.029442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:27.073962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99059f3c-a4f4-4c36-9cd7-413469ba6a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:27.075108 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99059f3c-a4f4-4c36-9cd7-413469ba6a7d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:27.076411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99059f3c-a4f4-4c36-9cd7-413469ba6a7d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:27.077045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99059f3c-a4f4-4c36-9cd7-413469ba6a7d None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:27.079598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99059f3c-a4f4-4c36-9cd7-413469ba6a7d None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:27.084749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99059f3c-a4f4-4c36-9cd7-413469ba6a7d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:27.088840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 149/291] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:26 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:27.093021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99059f3c-a4f4-4c36-9cd7-413469ba6a7d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:27.123400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f369b-bebf-43e2-b550-ae9f6f5ba642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:27.124401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f369b-bebf-43e2-b550-ae9f6f5ba642 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:27.124877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f369b-bebf-43e2-b550-ae9f6f5ba642 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:27.125366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f369b-bebf-43e2-b550-ae9f6f5ba642 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:27.126686 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f369b-bebf-43e2-b550-ae9f6f5ba642 None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:27.129049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f369b-bebf-43e2-b550-ae9f6f5ba642 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:27.131893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 143/292] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:26 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:27.137818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f369b-bebf-43e2-b550-ae9f6f5ba642 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:27.152261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:27.272269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99059f3c-a4f4-4c36-9cd7-413469ba6a7d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:27.327793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f369b-bebf-43e2-b550-ae9f6f5ba642 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:27.332620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b58eb-fb2a-4d1d-a1b0-26a5de116495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:27.333532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b58eb-fb2a-4d1d-a1b0-26a5de116495 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:27.334011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b58eb-fb2a-4d1d-a1b0-26a5de116495 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:27.334477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b58eb-fb2a-4d1d-a1b0-26a5de116495 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:27.335396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b58eb-fb2a-4d1d-a1b0-26a5de116495 None admin] RBAC: Adding query filter params (role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, scope.project.id=a36c670998e84f8ba575f12eebf55432, user.id=6127b7c574304a3d9a61e4caf2922003) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:27.335916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b58eb-fb2a-4d1d-a1b0-26a5de116495 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:27.337590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b58eb-fb2a-4d1d-a1b0-26a5de116495 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:27.356546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 150/293] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:27 2022] GET /identity/v3/role_assignments?user.id=6127b7c574304a3d9a61e4caf2922003&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:27.387540 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f9a847-5c86-4ea1-8b22-379eb643938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:27.388301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9a847-5c86-4ea1-8b22-379eb643938d None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:27.388599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9a847-5c86-4ea1-8b22-379eb643938d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:27.389041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f9a847-5c86-4ea1-8b22-379eb643938d None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432/users/bcee039de8d54d1f82ec9c49f7b5e692/roles/1f60cad77a004587a328e9dff0cae088` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:27.394675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9a847-5c86-4ea1-8b22-379eb643938d None admin] RBAC: Authorizing `identity:create_grant(project_id=a36c670998e84f8ba575f12eebf55432, user_id=bcee039de8d54d1f82ec9c49f7b5e692, role_id=1f60cad77a004587a328e9dff0cae088)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:27.396142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f9a847-5c86-4ea1-8b22-379eb643938d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:27.427713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 144/294] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:56:27 2022] PUT /identity/v3/projects/a36c670998e84f8ba575f12eebf55432/users/bcee039de8d54d1f82ec9c49f7b5e692/roles/1f60cad77a004587a328e9dff0cae088 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:27.848824 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859edf6-b7d3-4057-bba7-ecb1e71305ba None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:27.849356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859edf6-b7d3-4057-bba7-ecb1e71305ba None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:27.849432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859edf6-b7d3-4057-bba7-ecb1e71305ba None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:27.851043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 151/295] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:27.856512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429561-65d8-43d5-8e2a-4ac64bdaeaaa None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:27.856792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429561-65d8-43d5-8e2a-4ac64bdaeaaa None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:27.857633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9429561-65d8-43d5-8e2a-4ac64bdaeaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:27.878394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e9429561-65d8-43d5-8e2a-4ac64bdaeaaa None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:27.879600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e9429561-65d8-43d5-8e2a-4ac64bdaeaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:27.962677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:27.974860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 145/296] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:28.010960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25306823-6f3a-40d7-bbf5-d9ae18713042 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:28.011262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25306823-6f3a-40d7-bbf5-d9ae18713042 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:28.011502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25306823-6f3a-40d7-bbf5-d9ae18713042 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:28.012973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 152/297] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:28.019196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b39895-06e3-4fe2-bab1-db58e8a263e5 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:28.019483 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b39895-06e3-4fe2-bab1-db58e8a263e5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:28.019756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b39895-06e3-4fe2-bab1-db58e8a263e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:28.040957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-86b39895-06e3-4fe2-bab1-db58e8a263e5 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:28.042147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-86b39895-06e3-4fe2-bab1-db58e8a263e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:28.086901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882c0f7-8e95-4438-a6a7-a26ef55fa829 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:28.087179 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882c0f7-8e95-4438-a6a7-a26ef55fa829 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:28.087454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6882c0f7-8e95-4438-a6a7-a26ef55fa829 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:28.089013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 153/298] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:28.094968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec8591-389f-4506-bf28-98a96f68300e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:28.095262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec8591-389f-4506-bf28-98a96f68300e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:28.095526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec8591-389f-4506-bf28-98a96f68300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:28.116730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-14ec8591-389f-4506-bf28-98a96f68300e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:28.117870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-14ec8591-389f-4506-bf28-98a96f68300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:28.119268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 146/299] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:28.140709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:28.194687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 154/300] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:28.274414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b39895-06e3-4fe2-bab1-db58e8a263e5 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:28.375917 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b39895-06e3-4fe2-bab1-db58e8a263e5 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:28.392025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec8591-389f-4506-bf28-98a96f68300e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:28.406353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:28.416843 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0f4f1-212a-4841-96c7-902db24dd6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:28.418022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0f4f1-212a-4841-96c7-902db24dd6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:28.418294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0f4f1-212a-4841-96c7-902db24dd6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:28.418563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0f4f1-212a-4841-96c7-902db24dd6a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:28.436816 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0f4f1-212a-4841-96c7-902db24dd6a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-cee0f4f1-212a-4841-96c7-902db24dd6a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:28.441904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:28.442794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:28.443610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 147/301] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:28.448702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0f4f1-212a-4841-96c7-902db24dd6a0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:28.516508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec8591-389f-4506-bf28-98a96f68300e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:28.551684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0f4f1-212a-4841-96c7-902db24dd6a0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:28.556375 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a6976-b0e3-425d-8a87-61ae16b55d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:28.557233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a6976-b0e3-425d-8a87-61ae16b55d1e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:28.557681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a6976-b0e3-425d-8a87-61ae16b55d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:28.558118 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a6976-b0e3-425d-8a87-61ae16b55d1e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:28.571282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928a6976-b0e3-425d-8a87-61ae16b55d1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-928a6976-b0e3-425d-8a87-61ae16b55d1e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:28.577264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:28.578211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:28.578996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:28.580910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 155/302] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:28.581822 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a6976-b0e3-425d-8a87-61ae16b55d1e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:28.592950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852b674-3a15-4ce4-8e76-8a797491e1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:28.593577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b674-3a15-4ce4-8e76-8a797491e1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:28.593857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b674-3a15-4ce4-8e76-8a797491e1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:28.594120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b852b674-3a15-4ce4-8e76-8a797491e1a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:28.594771 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852b674-3a15-4ce4-8e76-8a797491e1a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:28.595067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852b674-3a15-4ce4-8e76-8a797491e1a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:28.596486 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b852b674-3a15-4ce4-8e76-8a797491e1a7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:28.601162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:28.604907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 148/303] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:28.605648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852b674-3a15-4ce4-8e76-8a797491e1a7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:28.655081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:28.697318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a6976-b0e3-425d-8a87-61ae16b55d1e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:28.708342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b852b674-3a15-4ce4-8e76-8a797491e1a7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:28.734775 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5974c-96ed-4582-a20c-8c3d6647b2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:28.735710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5974c-96ed-4582-a20c-8c3d6647b2ef None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:28.735990 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5974c-96ed-4582-a20c-8c3d6647b2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:28.736282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5974c-96ed-4582-a20c-8c3d6647b2ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:28.737410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5974c-96ed-4582-a20c-8c3d6647b2ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:28.738818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5974c-96ed-4582-a20c-8c3d6647b2ef None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:28.740374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 156/304] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:28.741133 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5974c-96ed-4582-a20c-8c3d6647b2ef None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:28.749554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cbe9f-e433-4937-8b6d-2fd38f265bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:28.750168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cbe9f-e433-4937-8b6d-2fd38f265bd6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:28.750444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cbe9f-e433-4937-8b6d-2fd38f265bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:28.750712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122cbe9f-e433-4937-8b6d-2fd38f265bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:28.751365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cbe9f-e433-4937-8b6d-2fd38f265bd6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:28.751667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cbe9f-e433-4937-8b6d-2fd38f265bd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:28.753460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122cbe9f-e433-4937-8b6d-2fd38f265bd6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:28.768254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 149/305] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:28.768254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cbe9f-e433-4937-8b6d-2fd38f265bd6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:28.830874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:28.848419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5974c-96ed-4582-a20c-8c3d6647b2ef None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:28.869520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122cbe9f-e433-4937-8b6d-2fd38f265bd6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:28.880545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:28.886240 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece8e13-c209-4689-a25b-a0fc14f4b778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:28.886881 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece8e13-c209-4689-a25b-a0fc14f4b778 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:28.887269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece8e13-c209-4689-a25b-a0fc14f4b778 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:28.887534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ece8e13-c209-4689-a25b-a0fc14f4b778 None admin] PATH_INFO: `/v3/users/08c4ee06f486499081496f150e44975e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:28.904191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece8e13-c209-4689-a25b-a0fc14f4b778 None admin] RBAC: Authorizing `identity:get_user(user_id=08c4ee06f486499081496f150e44975e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:28.904879 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e48c8-ea78-4330-ab15-bb56b623a90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:28.905571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e48c8-ea78-4330-ab15-bb56b623a90d None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:28.905737 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ece8e13-c209-4689-a25b-a0fc14f4b778 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:28.905842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e48c8-ea78-4330-ab15-bb56b623a90d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:28.906042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10e48c8-ea78-4330-ab15-bb56b623a90d None admin] PATH_INFO: `/v3/groups` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:28.906663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e48c8-ea78-4330-ab15-bb56b623a90d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:28.912040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 157/306] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:28 2022] GET /identity/v3/users/08c4ee06f486499081496f150e44975e => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:28.912158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10e48c8-ea78-4330-ab15-bb56b623a90d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:28.912251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece8e13-c209-4689-a25b-a0fc14f4b778 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:28.937142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 150/307] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Sep 22 21:56:28 2022] POST /identity/v3/groups => generated 229 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:56:28.937142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e48c8-ea78-4330-ab15-bb56b623a90d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:29.066795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10e48c8-ea78-4330-ab15-bb56b623a90d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:29.067796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ece8e13-c209-4689-a25b-a0fc14f4b778 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:29.105591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5950f-013c-41ed-911a-1c5eb3a8ca3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:29.106508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5950f-013c-41ed-911a-1c5eb3a8ca3d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:29.107678 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5950f-013c-41ed-911a-1c5eb3a8ca3d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:29.108292 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d5950f-013c-41ed-911a-1c5eb3a8ca3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:29.109313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5950f-013c-41ed-911a-1c5eb3a8ca3d None admin] RBAC: Adding query filter params (role.id=1f60cad77a004587a328e9dff0cae088, user.id=08c4ee06f486499081496f150e44975e, scope.system=all) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:29.109822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5950f-013c-41ed-911a-1c5eb3a8ca3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:29.111454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d5950f-013c-41ed-911a-1c5eb3a8ca3d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:29.118186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e901dc7-7a68-4b83-93de-1ac54e2dbec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:29.118333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901dc7-7a68-4b83-93de-1ac54e2dbec3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:29.118529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901dc7-7a68-4b83-93de-1ac54e2dbec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:29.118782 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e901dc7-7a68-4b83-93de-1ac54e2dbec3 None admin] PATH_INFO: `/v3/users/18273b3dc79a4c65bff906bf80f41a8b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:29.120079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e901dc7-7a68-4b83-93de-1ac54e2dbec3 None admin] RBAC: Authorizing `identity:get_user(user_id=18273b3dc79a4c65bff906bf80f41a8b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:29.121029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 151/308] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 22 21:56:28 2022] GET /identity/v3/role_assignments?user.id=08c4ee06f486499081496f150e44975e&scope.system=all&role.id=1f60cad77a004587a328e9dff0cae088 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:29.121942 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e901dc7-7a68-4b83-93de-1ac54e2dbec3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:29.125139 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 158/309] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:28 2022] GET /identity/v3/users/18273b3dc79a4c65bff906bf80f41a8b => generated 293 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:29.129512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5950f-013c-41ed-911a-1c5eb3a8ca3d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:29.175469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:29.257789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d5950f-013c-41ed-911a-1c5eb3a8ca3d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:29.301003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ab90e-b816-4e32-90e8-faee6531f959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:29.301705 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab90e-b816-4e32-90e8-faee6531f959 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:29.301982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab90e-b816-4e32-90e8-faee6531f959 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:29.302254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab90e-b816-4e32-90e8-faee6531f959 None admin] PATH_INFO: `/v3/system/users/18273b3dc79a4c65bff906bf80f41a8b/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:29.303709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ab90e-b816-4e32-90e8-faee6531f959 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=18273b3dc79a4c65bff906bf80f41a8b, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:29.305211 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ab90e-b816-4e32-90e8-faee6531f959 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:29.317476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 152/310] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Sep 22 21:56:29 2022] PUT /identity/v3/system/users/18273b3dc79a4c65bff906bf80f41a8b/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:29.885186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ff1f-5d46-488a-80af-c77fa0a25a03 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:29.885582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ff1f-5d46-488a-80af-c77fa0a25a03 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:29.885755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8111ff1f-5d46-488a-80af-c77fa0a25a03 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:29.887341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 159/311] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:29.892828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ecc30-c57d-410c-9bfa-4f6a4625c631 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:29.893244 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ecc30-c57d-410c-9bfa-4f6a4625c631 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:29.893430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ecc30-c57d-410c-9bfa-4f6a4625c631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:29.912332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d13ecc30-c57d-410c-9bfa-4f6a4625c631 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:29.913543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d13ecc30-c57d-410c-9bfa-4f6a4625c631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:29.987284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 153/312] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:30.257464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ff1f-5d46-488a-80af-c77fa0a25a03 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:30.270099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:30.364300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8111ff1f-5d46-488a-80af-c77fa0a25a03 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:30.399436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddeb619-5d64-425d-a656-4a021bd1b542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:30.400322 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddeb619-5d64-425d-a656-4a021bd1b542 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:30.400735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddeb619-5d64-425d-a656-4a021bd1b542 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:30.401165 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddeb619-5d64-425d-a656-4a021bd1b542 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:30.412432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddeb619-5d64-425d-a656-4a021bd1b542 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2ddeb619-5d64-425d-a656-4a021bd1b542 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:30.417539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:30.418578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:30.420064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:30.422180 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 160/313] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:30.433156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ecc30-c57d-410c-9bfa-4f6a4625c631 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:30.456968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:30.536065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13ecc30-c57d-410c-9bfa-4f6a4625c631 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:30.571205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308381f-18ba-442d-9012-7646e8f06c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:30.571812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308381f-18ba-442d-9012-7646e8f06c04 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:30.572090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308381f-18ba-442d-9012-7646e8f06c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:30.572364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9308381f-18ba-442d-9012-7646e8f06c04 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:30.573020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308381f-18ba-442d-9012-7646e8f06c04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:30.573349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308381f-18ba-442d-9012-7646e8f06c04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:30.574721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9308381f-18ba-442d-9012-7646e8f06c04 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:30.583558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 154/314] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:30.589244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddeb619-5d64-425d-a656-4a021bd1b542 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:30.665369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:30.697225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddeb619-5d64-425d-a656-4a021bd1b542 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:30.733195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb328ba-c696-4171-a489-4b9ecabc415b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:30.734202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb328ba-c696-4171-a489-4b9ecabc415b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:30.734669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb328ba-c696-4171-a489-4b9ecabc415b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:30.735130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb328ba-c696-4171-a489-4b9ecabc415b None admin] PATH_INFO: `/v3/users/bcee039de8d54d1f82ec9c49f7b5e692` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:30.736433 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb328ba-c696-4171-a489-4b9ecabc415b None admin] RBAC: Authorizing `identity:get_user(user_id=bcee039de8d54d1f82ec9c49f7b5e692)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:30.738085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb328ba-c696-4171-a489-4b9ecabc415b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:30.740305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 161/315] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:30 2022] GET /identity/v3/users/bcee039de8d54d1f82ec9c49f7b5e692 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:30.745991 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308381f-18ba-442d-9012-7646e8f06c04 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:30.845007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:30.856436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9308381f-18ba-442d-9012-7646e8f06c04 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:30.893550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb65090-bdf4-4b10-b754-085144427319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:30.894168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb65090-bdf4-4b10-b754-085144427319 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:30.894416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb65090-bdf4-4b10-b754-085144427319 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:30.894689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb65090-bdf4-4b10-b754-085144427319 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:30.966922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb65090-bdf4-4b10-b754-085144427319 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:30.968729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb65090-bdf4-4b10-b754-085144427319 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:31.073685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:31.087656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 155/316] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:31.097241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb328ba-c696-4171-a489-4b9ecabc415b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:31.206563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb328ba-c696-4171-a489-4b9ecabc415b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:31.247991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4688f49e-f622-4240-bfb3-a42a0070e861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:31.248653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688f49e-f622-4240-bfb3-a42a0070e861 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:31.248924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688f49e-f622-4240-bfb3-a42a0070e861 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:31.249225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4688f49e-f622-4240-bfb3-a42a0070e861 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:31.250453 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688f49e-f622-4240-bfb3-a42a0070e861 None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:31.251918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4688f49e-f622-4240-bfb3-a42a0070e861 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:31.254061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 162/317] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:31 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:31.259906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb65090-bdf4-4b10-b754-085144427319 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:31.345588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27afd-5ecf-4e07-8224-196670043452 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:31.345896 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27afd-5ecf-4e07-8224-196670043452 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:31.346124 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27afd-5ecf-4e07-8224-196670043452 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:31.347611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 163/318] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:31.352846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f899c0-47a0-435c-94d2-1f820906aa4b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:31.353154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f899c0-47a0-435c-94d2-1f820906aa4b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:31.353428 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f899c0-47a0-435c-94d2-1f820906aa4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:31.367168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb65090-bdf4-4b10-b754-085144427319 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:31.367498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:31.374969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-52f899c0-47a0-435c-94d2-1f820906aa4b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:31.376683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-52f899c0-47a0-435c-94d2-1f820906aa4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:31.408366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a112b46-182c-487a-9a80-666b54b37301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:31.409929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a112b46-182c-487a-9a80-666b54b37301 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:31.410411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a112b46-182c-487a-9a80-666b54b37301 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:31.410881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a112b46-182c-487a-9a80-666b54b37301 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:31.411790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a112b46-182c-487a-9a80-666b54b37301 None admin] RBAC: Adding query filter params (role.id=1f60cad77a004587a328e9dff0cae088, scope.project.id=a36c670998e84f8ba575f12eebf55432, user.id=bcee039de8d54d1f82ec9c49f7b5e692) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:31.412364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a112b46-182c-487a-9a80-666b54b37301 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:31.414995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a112b46-182c-487a-9a80-666b54b37301 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:31.428373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 156/319] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:56:31 2022] GET /identity/v3/role_assignments?user.id=bcee039de8d54d1f82ec9c49f7b5e692&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=1f60cad77a004587a328e9dff0cae088 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:31.428373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e383323-bc00-4603-9d78-26c8813cee5a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:31.428373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e383323-bc00-4603-9d78-26c8813cee5a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:31.428373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e383323-bc00-4603-9d78-26c8813cee5a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:31.428373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 157/320] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:31.430858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549d872-01ee-46c5-8634-15f7fb8b57f0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:31.431138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549d872-01ee-46c5-8634-15f7fb8b57f0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:31.431663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3549d872-01ee-46c5-8634-15f7fb8b57f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:31.451635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3549d872-01ee-46c5-8634-15f7fb8b57f0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:31.452855 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3549d872-01ee-46c5-8634-15f7fb8b57f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:31.469383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 164/321] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:31.523863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:31.540381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 158/322] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:31.593542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6126f-0352-4392-a983-ed17109dacc0 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:31.594057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6126f-0352-4392-a983-ed17109dacc0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:31.594503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6126f-0352-4392-a983-ed17109dacc0 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:31.596307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 165/323] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:31.602240 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e3bb0-213f-45b2-aaac-ca97f0e0251c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:31.602704 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e3bb0-213f-45b2-aaac-ca97f0e0251c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:31.603142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e3bb0-213f-45b2-aaac-ca97f0e0251c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:31.622246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-bd1e3bb0-213f-45b2-aaac-ca97f0e0251c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:31.624207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bd1e3bb0-213f-45b2-aaac-ca97f0e0251c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:31.696775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 159/324] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:31.768477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6126f-0352-4392-a983-ed17109dacc0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:31.814935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e3bb0-213f-45b2-aaac-ca97f0e0251c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:31.871535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b6126f-0352-4392-a983-ed17109dacc0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:31.906510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edf855-a5a0-48ff-9e0c-efaec6313876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:31.907383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edf855-a5a0-48ff-9e0c-efaec6313876 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:31.907828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edf855-a5a0-48ff-9e0c-efaec6313876 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:31.908265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07edf855-a5a0-48ff-9e0c-efaec6313876 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:31.909573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edf855-a5a0-48ff-9e0c-efaec6313876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:31.911160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07edf855-a5a0-48ff-9e0c-efaec6313876 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:31.913240 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 166/325] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:31.918088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edf855-a5a0-48ff-9e0c-efaec6313876 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:31.921715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:31.922907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e3bb0-213f-45b2-aaac-ca97f0e0251c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:31.937611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:31.958057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef6d7f-12f0-45e9-8e49-8d35580ac8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:31.958682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6d7f-12f0-45e9-8e49-8d35580ac8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:31.958956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6d7f-12f0-45e9-8e49-8d35580ac8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:31.959219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef6d7f-12f0-45e9-8e49-8d35580ac8a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:31.973152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef6d7f-12f0-45e9-8e49-8d35580ac8a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-72ef6d7f-12f0-45e9-8e49-8d35580ac8a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:31.974762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:31.975729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:31.976553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 160/326] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:31.977226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef6d7f-12f0-45e9-8e49-8d35580ac8a7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:32.026592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07edf855-a5a0-48ff-9e0c-efaec6313876 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:32.063209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b680e98-d43b-4d26-9a41-e21ab6918fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:32.063816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b680e98-d43b-4d26-9a41-e21ab6918fe6 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:32.064090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b680e98-d43b-4d26-9a41-e21ab6918fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:32.064364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b680e98-d43b-4d26-9a41-e21ab6918fe6 None admin] PATH_INFO: `/v3/groups` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:32.064989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b680e98-d43b-4d26-9a41-e21ab6918fe6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:32.066547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b680e98-d43b-4d26-9a41-e21ab6918fe6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:32.078499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef6d7f-12f0-45e9-8e49-8d35580ac8a7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:32.083355 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 167/327] 10.222.0.25 () {60 vars in 1134 bytes} [Thu Sep 22 21:56:31 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:56:32.084064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b680e98-d43b-4d26-9a41-e21ab6918fe6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:32.120613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf275df3-b4c3-4c13-bcfe-fe5c4cea60c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:32.121703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf275df3-b4c3-4c13-bcfe-fe5c4cea60c2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:32.122754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf275df3-b4c3-4c13-bcfe-fe5c4cea60c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:32.123207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf275df3-b4c3-4c13-bcfe-fe5c4cea60c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:32.135227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf275df3-b4c3-4c13-bcfe-fe5c4cea60c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-bf275df3-b4c3-4c13-bcfe-fe5c4cea60c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:32.141285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:32.142237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:32.143020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:32.145688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 161/328] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:32.151524 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf275df3-b4c3-4c13-bcfe-fe5c4cea60c2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:32.173359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:32.195394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:32.219087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b680e98-d43b-4d26-9a41-e21ab6918fe6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:32.259479 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee87962-36d4-4333-9994-012849d04cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:32.260323 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee87962-36d4-4333-9994-012849d04cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:32.260401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee87962-36d4-4333-9994-012849d04cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:32.260634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee87962-36d4-4333-9994-012849d04cf8 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:32.261322 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee87962-36d4-4333-9994-012849d04cf8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:32.262247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee87962-36d4-4333-9994-012849d04cf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:32.263625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee87962-36d4-4333-9994-012849d04cf8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:32.273141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 168/329] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:32.276746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf275df3-b4c3-4c13-bcfe-fe5c4cea60c2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:32.278570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee87962-36d4-4333-9994-012849d04cf8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:32.315505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed4d33-8e39-4e16-badd-9ef9ee1482dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:32.316448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed4d33-8e39-4e16-badd-9ef9ee1482dd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:32.316448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed4d33-8e39-4e16-badd-9ef9ee1482dd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:32.316722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eed4d33-8e39-4e16-badd-9ef9ee1482dd None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:32.317373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed4d33-8e39-4e16-badd-9ef9ee1482dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:32.318721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed4d33-8e39-4e16-badd-9ef9ee1482dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:32.319273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eed4d33-8e39-4e16-badd-9ef9ee1482dd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:32.329931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 162/330] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:32.335741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed4d33-8e39-4e16-badd-9ef9ee1482dd None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:32.413842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee87962-36d4-4333-9994-012849d04cf8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:32.445798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:32.458222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9b0a1-fb1f-47e2-890e-3e3c32d23ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:32.458868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9b0a1-fb1f-47e2-890e-3e3c32d23ffb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:32.459150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9b0a1-fb1f-47e2-890e-3e3c32d23ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:32.459426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9b0a1-fb1f-47e2-890e-3e3c32d23ffb None admin] PATH_INFO: `/v3/users/08c4ee06f486499081496f150e44975e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:32.459893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:32.460619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9b0a1-fb1f-47e2-890e-3e3c32d23ffb None admin] RBAC: Authorizing `identity:get_user(user_id=08c4ee06f486499081496f150e44975e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:32.462470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9b0a1-fb1f-47e2-890e-3e3c32d23ffb None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:32.464247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 169/331] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:32 2022] GET /identity/v3/users/08c4ee06f486499081496f150e44975e => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:32.468697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eed4d33-8e39-4e16-badd-9ef9ee1482dd None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:32.471058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9b0a1-fb1f-47e2-890e-3e3c32d23ffb None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:32.508189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa86deb-179f-4cd9-aa02-58598a60460a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:32.508794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa86deb-179f-4cd9-aa02-58598a60460a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:32.509080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa86deb-179f-4cd9-aa02-58598a60460a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:32.509408 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa86deb-179f-4cd9-aa02-58598a60460a None admin] PATH_INFO: `/v3/users/18273b3dc79a4c65bff906bf80f41a8b` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:32.510649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa86deb-179f-4cd9-aa02-58598a60460a None admin] RBAC: Authorizing `identity:get_user(user_id=18273b3dc79a4c65bff906bf80f41a8b)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:32.512095 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa86deb-179f-4cd9-aa02-58598a60460a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:32.514219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 163/332] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:32 2022] GET /identity/v3/users/18273b3dc79a4c65bff906bf80f41a8b => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:32.519594 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa86deb-179f-4cd9-aa02-58598a60460a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:32.602280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9b0a1-fb1f-47e2-890e-3e3c32d23ffb None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:32.644030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d594a-99a5-4f3a-9eeb-4436826e1c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:32.644952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d594a-99a5-4f3a-9eeb-4436826e1c32 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:32.645277 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d594a-99a5-4f3a-9eeb-4436826e1c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:32.645549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d594a-99a5-4f3a-9eeb-4436826e1c32 None admin] PATH_INFO: `/v3/system/users/08c4ee06f486499081496f150e44975e/roles/1f60cad77a004587a328e9dff0cae088` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:32.646975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d594a-99a5-4f3a-9eeb-4436826e1c32 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=08c4ee06f486499081496f150e44975e, role_id=1f60cad77a004587a328e9dff0cae088)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:32.648363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d594a-99a5-4f3a-9eeb-4436826e1c32 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:32.649924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa86deb-179f-4cd9-aa02-58598a60460a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:32.664129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 170/333] 10.222.0.25 () {58 vars in 1334 bytes} [Thu Sep 22 21:56:32 2022] PUT /identity/v3/system/users/08c4ee06f486499081496f150e44975e/roles/1f60cad77a004587a328e9dff0cae088 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 21:56:32.696572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295483af-0eb9-436c-8330-5a6b637680e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:32.697287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295483af-0eb9-436c-8330-5a6b637680e4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:32.697567 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295483af-0eb9-436c-8330-5a6b637680e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:32.697837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295483af-0eb9-436c-8330-5a6b637680e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:32.698584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295483af-0eb9-436c-8330-5a6b637680e4 None admin] RBAC: Adding query filter params (role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, user.id=18273b3dc79a4c65bff906bf80f41a8b, scope.system=all) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:32.698890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295483af-0eb9-436c-8330-5a6b637680e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:32.700538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295483af-0eb9-436c-8330-5a6b637680e4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:32.717673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 164/334] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 22 21:56:32 2022] GET /identity/v3/role_assignments?user.id=18273b3dc79a4c65bff906bf80f41a8b&scope.system=all&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:34.720372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ee91a-85bd-4a19-aae1-3d5082f7ccad None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:34.721455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ee91a-85bd-4a19-aae1-3d5082f7ccad None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:34.721950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0ee91a-85bd-4a19-aae1-3d5082f7ccad None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:34.723815 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 171/335] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:34.729023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697298e6-1111-4b9e-8ec7-ffa18842c4bb None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:34.729899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697298e6-1111-4b9e-8ec7-ffa18842c4bb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:34.730375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697298e6-1111-4b9e-8ec7-ffa18842c4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:34.758121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-697298e6-1111-4b9e-8ec7-ffa18842c4bb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:34.759721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-697298e6-1111-4b9e-8ec7-ffa18842c4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:34.773812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70996e3-dcef-4275-9dbf-3f862e86fbf1 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:34.774413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70996e3-dcef-4275-9dbf-3f862e86fbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:34.774919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70996e3-dcef-4275-9dbf-3f862e86fbf1 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:34.776729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 172/336] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:34.783228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b5e6d-400e-4486-815f-281692ea8526 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:34.783710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b5e6d-400e-4486-815f-281692ea8526 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:34.784135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b5e6d-400e-4486-815f-281692ea8526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:34.787536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:34.826743 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8d0b5e6d-400e-4486-815f-281692ea8526 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:34.828326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8d0b5e6d-400e-4486-815f-281692ea8526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:34.870727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 165/337] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:34.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:34.945929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 173/338] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:35.206352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4d3c-eb57-4f64-a921-c6123df37396 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:35.206984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4d3c-eb57-4f64-a921-c6123df37396 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:35.207493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d4d3c-eb57-4f64-a921-c6123df37396 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:35.209028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 166/339] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:35.214465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b497-0452-48ce-b8dd-05436aed4763 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:35.214744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b497-0452-48ce-b8dd-05436aed4763 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:35.215012 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4074b497-0452-48ce-b8dd-05436aed4763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:35.240339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4074b497-0452-48ce-b8dd-05436aed4763 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:35.241546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4074b497-0452-48ce-b8dd-05436aed4763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:35.321169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 174/340] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:35.354154 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d4d3c-eb57-4f64-a921-c6123df37396 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:35.482170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d4d3c-eb57-4f64-a921-c6123df37396 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:35.532241 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce8cef-8d53-443c-ae5f-2125c68ce0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:35.532903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce8cef-8d53-443c-ae5f-2125c68ce0b6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:35.533211 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce8cef-8d53-443c-ae5f-2125c68ce0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:35.533488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce8cef-8d53-443c-ae5f-2125c68ce0b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:35.553802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce8cef-8d53-443c-ae5f-2125c68ce0b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-06ce8cef-8d53-443c-ae5f-2125c68ce0b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:35.558745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:35.560265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:35.561627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 167/341] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:35.567088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074b497-0452-48ce-b8dd-05436aed4763 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:35.614408 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce8cef-8d53-443c-ae5f-2125c68ce0b6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:35.644338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:35.692808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4074b497-0452-48ce-b8dd-05436aed4763 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:35.716487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:35.746326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c55c21-20ca-427c-b048-24cd06e7dade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:35.748295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c55c21-20ca-427c-b048-24cd06e7dade None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:35.748766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c55c21-20ca-427c-b048-24cd06e7dade None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:35.749249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c55c21-20ca-427c-b048-24cd06e7dade None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:35.750108 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c55c21-20ca-427c-b048-24cd06e7dade None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:35.750615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c55c21-20ca-427c-b048-24cd06e7dade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:35.752160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c55c21-20ca-427c-b048-24cd06e7dade None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:35.764819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 175/342] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:35.766022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c55c21-20ca-427c-b048-24cd06e7dade None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:35.791070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce8cef-8d53-443c-ae5f-2125c68ce0b6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:35.846578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e001484-6ff7-44e2-874b-d81957873463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:35.847526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e001484-6ff7-44e2-874b-d81957873463 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:35.848509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e001484-6ff7-44e2-874b-d81957873463 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:35.848955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e001484-6ff7-44e2-874b-d81957873463 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:35.864999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e001484-6ff7-44e2-874b-d81957873463 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-7e001484-6ff7-44e2-874b-d81957873463 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:35.869776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:35.870631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:35.871416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 168/343] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:35.872596 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48458cca-7947-4e04-90b7-efee364ae5e9 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:35.872851 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48458cca-7947-4e04-90b7-efee364ae5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:35.876281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48458cca-7947-4e04-90b7-efee364ae5e9 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:35.876281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 169/344] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:35.876281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48458cca-7947-4e04-90b7-efee364ae5e9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:35.905298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c55c21-20ca-427c-b048-24cd06e7dade None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:35.954005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c3f5c-051d-47a0-baba-51ff1eb3f5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:35.954673 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c3f5c-051d-47a0-baba-51ff1eb3f5cb None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:35.954950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c3f5c-051d-47a0-baba-51ff1eb3f5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:35.956202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c3f5c-051d-47a0-baba-51ff1eb3f5cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:35.970274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c3f5c-051d-47a0-baba-51ff1eb3f5cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:35.974888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d12c3f5c-051d-47a0-baba-51ff1eb3f5cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:35.976201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:35.977236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:35.978065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 176/345] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:35.979464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e414c-894e-4385-9ef6-15961295a99c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:35.979464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e414c-894e-4385-9ef6-15961295a99c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:35.979464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e414c-894e-4385-9ef6-15961295a99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:35.999955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3e7e414c-894e-4385-9ef6-15961295a99c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:36.001180 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3e7e414c-894e-4385-9ef6-15961295a99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:36.019396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48458cca-7947-4e04-90b7-efee364ae5e9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:36.042671 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:36.079254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3bc99-9e2e-4e00-be9e-e2f9fee76f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:36.080248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3bc99-9e2e-4e00-be9e-e2f9fee76f30 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:36.080721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3bc99-9e2e-4e00-be9e-e2f9fee76f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:36.081315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3bc99-9e2e-4e00-be9e-e2f9fee76f30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:36.098432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 177/346] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:36.099386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e414c-894e-4385-9ef6-15961295a99c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:36.171221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3bc99-9e2e-4e00-be9e-e2f9fee76f30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:36.173001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f3bc99-9e2e-4e00-be9e-e2f9fee76f30 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:36.245465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:36.278472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 170/347] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:36.279492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3bc99-9e2e-4e00-be9e-e2f9fee76f30 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:36.299030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e414c-894e-4385-9ef6-15961295a99c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:36.383372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3bc99-9e2e-4e00-be9e-e2f9fee76f30 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:36.393821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee891ff3-46cd-46e2-a04e-1dd02b19b1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:36.394449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee891ff3-46cd-46e2-a04e-1dd02b19b1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:36.394727 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee891ff3-46cd-46e2-a04e-1dd02b19b1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:36.394999 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee891ff3-46cd-46e2-a04e-1dd02b19b1c1 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:36.395666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee891ff3-46cd-46e2-a04e-1dd02b19b1c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:36.395963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee891ff3-46cd-46e2-a04e-1dd02b19b1c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:36.397363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee891ff3-46cd-46e2-a04e-1dd02b19b1c1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:36.413496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 178/348] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:36.414335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee891ff3-46cd-46e2-a04e-1dd02b19b1c1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:36.422718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c51d36-4ccb-4b06-a492-4378c2550933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:36.423271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c51d36-4ccb-4b06-a492-4378c2550933 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:36.423671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c51d36-4ccb-4b06-a492-4378c2550933 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:36.423931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c51d36-4ccb-4b06-a492-4378c2550933 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:36.425284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c51d36-4ccb-4b06-a492-4378c2550933 None admin] RBAC: Adding query filter params (name=member) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:36.425284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c51d36-4ccb-4b06-a492-4378c2550933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:36.426539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c51d36-4ccb-4b06-a492-4378c2550933 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:36.435997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 171/349] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:36.436839 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c51d36-4ccb-4b06-a492-4378c2550933 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:36.473784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:36.490266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:36.547837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c51d36-4ccb-4b06-a492-4378c2550933 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:36.581594 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee891ff3-46cd-46e2-a04e-1dd02b19b1c1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:36.585056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d76e-35b9-4b68-b91c-fa875cebbd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:36.585056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d76e-35b9-4b68-b91c-fa875cebbd2e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:36.585268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d76e-35b9-4b68-b91c-fa875cebbd2e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:36.585535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418d76e-35b9-4b68-b91c-fa875cebbd2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:36.629045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aa7d3-7884-4d49-921a-219c1e74228a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:36.630103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aa7d3-7884-4d49-921a-219c1e74228a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:36.630584 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aa7d3-7884-4d49-921a-219c1e74228a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:36.631074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aa7d3-7884-4d49-921a-219c1e74228a None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:36.632460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aa7d3-7884-4d49-921a-219c1e74228a None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:36.634777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aa7d3-7884-4d49-921a-219c1e74228a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:36.638583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 179/350] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:36 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:36.640011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aa7d3-7884-4d49-921a-219c1e74228a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:36.673584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418d76e-35b9-4b68-b91c-fa875cebbd2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:36.675814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418d76e-35b9-4b68-b91c-fa875cebbd2e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:36.726143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:36.765192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:36.771264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aa7d3-7884-4d49-921a-219c1e74228a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:36.803602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 172/351] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:36.804453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d76e-35b9-4b68-b91c-fa875cebbd2e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:36.828280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdaf30-cfd6-43be-9929-a1abc6bae89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:36.829330 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdaf30-cfd6-43be-9929-a1abc6bae89a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:36.829814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdaf30-cfd6-43be-9929-a1abc6bae89a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:36.830300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bdaf30-cfd6-43be-9929-a1abc6bae89a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:36.921334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdaf30-cfd6-43be-9929-a1abc6bae89a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:36.923522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bdaf30-cfd6-43be-9929-a1abc6bae89a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:36.928552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418d76e-35b9-4b68-b91c-fa875cebbd2e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:36.976987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31c617-d330-411c-958b-98dd9a37926d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:36.977650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31c617-d330-411c-958b-98dd9a37926d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:36.977922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31c617-d330-411c-958b-98dd9a37926d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:36.978192 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b31c617-d330-411c-958b-98dd9a37926d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:36.982621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:36.994636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b31c617-d330-411c-958b-98dd9a37926d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-7b31c617-d330-411c-958b-98dd9a37926d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:36.999980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:37.000903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 173/352] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:37.002635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31c617-d330-411c-958b-98dd9a37926d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:37.081436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:37.090274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 180/353] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:37.091659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdaf30-cfd6-43be-9929-a1abc6bae89a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:37.148806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b31c617-d330-411c-958b-98dd9a37926d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:37.197811 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853d135-7487-4708-9900-fb7f4ccd8214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:37.198676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853d135-7487-4708-9900-fb7f4ccd8214 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.198954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853d135-7487-4708-9900-fb7f4ccd8214 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.199228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853d135-7487-4708-9900-fb7f4ccd8214 None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.214302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853d135-7487-4708-9900-fb7f4ccd8214 None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:37.217247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853d135-7487-4708-9900-fb7f4ccd8214 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:37.220242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 174/354] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:37 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:37.221837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853d135-7487-4708-9900-fb7f4ccd8214 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:37.230227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bdaf30-cfd6-43be-9929-a1abc6bae89a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:37.272469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8efd1f-5048-4921-9b50-43d9f75d327a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:37.273121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8efd1f-5048-4921-9b50-43d9f75d327a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.273469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8efd1f-5048-4921-9b50-43d9f75d327a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.273734 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8efd1f-5048-4921-9b50-43d9f75d327a None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.274861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8efd1f-5048-4921-9b50-43d9f75d327a None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:37.276310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8efd1f-5048-4921-9b50-43d9f75d327a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:37.278144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 181/355] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:37 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:37.280189 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad4485-b633-473b-b6cb-959062603c6b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.280470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad4485-b633-473b-b6cb-959062603c6b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.280739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad4485-b633-473b-b6cb-959062603c6b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.283882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 182/356] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:37.283882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad4485-b633-473b-b6cb-959062603c6b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:37.305824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:37.350456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853d135-7487-4708-9900-fb7f4ccd8214 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:37.365474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:37.387639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb7878-6a75-4489-8518-6ee249d1b1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:37.388582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb7878-6a75-4489-8518-6ee249d1b1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.388824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb7878-6a75-4489-8518-6ee249d1b1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.389083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb7878-6a75-4489-8518-6ee249d1b1d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.389780 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb7878-6a75-4489-8518-6ee249d1b1d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:37.390084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb7878-6a75-4489-8518-6ee249d1b1d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:37.391467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb7878-6a75-4489-8518-6ee249d1b1d0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:37.403835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 175/357] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:37.404480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb7878-6a75-4489-8518-6ee249d1b1d0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:37.406797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad4485-b633-473b-b6cb-959062603c6b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:37.444845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a3826-1d93-44f9-88d9-b7264bfd4e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:37.445522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a3826-1d93-44f9-88d9-b7264bfd4e1b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.445812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a3826-1d93-44f9-88d9-b7264bfd4e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.446068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a3826-1d93-44f9-88d9-b7264bfd4e1b None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.447924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a3826-1d93-44f9-88d9-b7264bfd4e1b None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:37.449356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a3826-1d93-44f9-88d9-b7264bfd4e1b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:37.451779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 183/358] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:37 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:37.453011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a3826-1d93-44f9-88d9-b7264bfd4e1b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:37.526047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb7878-6a75-4489-8518-6ee249d1b1d0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:37.546833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:37.571792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38240ed8-9719-42c8-bc93-a3844568b504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:37.572395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240ed8-9719-42c8-bc93-a3844568b504 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.572670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240ed8-9719-42c8-bc93-a3844568b504 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.572938 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38240ed8-9719-42c8-bc93-a3844568b504 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.573748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38240ed8-9719-42c8-bc93-a3844568b504 None admin] RBAC: Adding query filter params (group.id=ec1800a99f56492687b58a547023a1c3, role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, scope.project.id=8087b41e58fb4aca97cbfced2c011450) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:37.574190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38240ed8-9719-42c8-bc93-a3844568b504 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:37.575999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38240ed8-9719-42c8-bc93-a3844568b504 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:37.577552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a3826-1d93-44f9-88d9-b7264bfd4e1b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:37.582097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 176/359] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 22 21:56:37 2022] GET /identity/v3/role_assignments?group.id=ec1800a99f56492687b58a547023a1c3&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:37.584127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f9574-c26b-47db-bde0-dfce1177abe1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.584127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f9574-c26b-47db-bde0-dfce1177abe1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.584127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f9574-c26b-47db-bde0-dfce1177abe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.603288 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9c9f9574-c26b-47db-bde0-dfce1177abe1 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:37.604514 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9c9f9574-c26b-47db-bde0-dfce1177abe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:37.633097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643e349-f459-4e1a-bbeb-89b7c8b8269d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:37.633097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643e349-f459-4e1a-bbeb-89b7c8b8269d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.633097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643e349-f459-4e1a-bbeb-89b7c8b8269d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.633097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643e349-f459-4e1a-bbeb-89b7c8b8269d None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.634369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643e349-f459-4e1a-bbeb-89b7c8b8269d None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:37.636078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643e349-f459-4e1a-bbeb-89b7c8b8269d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:37.638909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 184/360] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:37 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:37.640483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d32c5-d7ca-4958-a95d-fb120c2505c3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.640836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d32c5-d7ca-4958-a95d-fb120c2505c3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.641226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d32c5-d7ca-4958-a95d-fb120c2505c3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.642946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 185/361] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:37.643815 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d32c5-d7ca-4958-a95d-fb120c2505c3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:37.654522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:37.693734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 177/362] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:37.694922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f9574-c26b-47db-bde0-dfce1177abe1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:37.795161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d32c5-d7ca-4958-a95d-fb120c2505c3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:37.817585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f9574-c26b-47db-bde0-dfce1177abe1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:37.833868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b9412-8431-4aca-bb63-4b0ad02cb376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:37.835538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:37.836218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b9412-8431-4aca-bb63-4b0ad02cb376 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.836326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b9412-8431-4aca-bb63-4b0ad02cb376 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.836555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b9412-8431-4aca-bb63-4b0ad02cb376 None admin] PATH_INFO: `/v3/users/08c4ee06f486499081496f150e44975e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.837768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b9412-8431-4aca-bb63-4b0ad02cb376 None admin] RBAC: Authorizing `identity:get_user(user_id=08c4ee06f486499081496f150e44975e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:37.839366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b9412-8431-4aca-bb63-4b0ad02cb376 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:37.841327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 186/363] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:56:37 2022] GET /identity/v3/users/08c4ee06f486499081496f150e44975e => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:37.842378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b9412-8431-4aca-bb63-4b0ad02cb376 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:37.862134 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5fe73c-4b32-4c91-b665-eb38cd0d027c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:37.862871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fe73c-4b32-4c91-b665-eb38cd0d027c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.863216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fe73c-4b32-4c91-b665-eb38cd0d027c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.863552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fe73c-4b32-4c91-b665-eb38cd0d027c None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.864714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5fe73c-4b32-4c91-b665-eb38cd0d027c None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:37.866323 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5fe73c-4b32-4c91-b665-eb38cd0d027c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:37.868456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 178/364] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:37 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:37.869671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a0c62-8a7a-429e-b86e-90be3deca078 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.869946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a0c62-8a7a-429e-b86e-90be3deca078 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.870215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a0c62-8a7a-429e-b86e-90be3deca078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.890341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-423a0c62-8a7a-429e-b86e-90be3deca078 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:37.891531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-423a0c62-8a7a-429e-b86e-90be3deca078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:37.920645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:37.959110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b9412-8431-4aca-bb63-4b0ad02cb376 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:37.995868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a152b3-8625-48be-a72f-11e3cef8ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:37.996528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a152b3-8625-48be-a72f-11e3cef8ca29 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:37.997449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a152b3-8625-48be-a72f-11e3cef8ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:37.997734 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a152b3-8625-48be-a72f-11e3cef8ca29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:37.998474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a152b3-8625-48be-a72f-11e3cef8ca29 None admin] RBAC: Adding query filter params (group.id=ec1800a99f56492687b58a547023a1c3, role.id=dea086603b04451680a14ae806205d79, scope.project.id=8087b41e58fb4aca97cbfced2c011450) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:37.998780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a152b3-8625-48be-a72f-11e3cef8ca29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:38.000211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a152b3-8625-48be-a72f-11e3cef8ca29 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:38.002691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 179/365] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:38.004561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a0c62-8a7a-429e-b86e-90be3deca078 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:38.006587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 187/366] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 22 21:56:37 2022] GET /identity/v3/role_assignments?group.id=ec1800a99f56492687b58a547023a1c3&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=dea086603b04451680a14ae806205d79 => generated 260 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:38.007331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a152b3-8625-48be-a72f-11e3cef8ca29 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:38.130266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a152b3-8625-48be-a72f-11e3cef8ca29 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:38.177921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b52a0-54b0-4b54-aa94-f34e6dfddfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:38.178610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b52a0-54b0-4b54-aa94-f34e6dfddfd5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:38.178891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b52a0-54b0-4b54-aa94-f34e6dfddfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:38.179165 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b52a0-54b0-4b54-aa94-f34e6dfddfd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:38.179912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b52a0-54b0-4b54-aa94-f34e6dfddfd5 None admin] RBAC: Adding query filter params (group.id=ec1800a99f56492687b58a547023a1c3, role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, scope.project.id=a36c670998e84f8ba575f12eebf55432) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:38.180225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b52a0-54b0-4b54-aa94-f34e6dfddfd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:38.181818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695b52a0-54b0-4b54-aa94-f34e6dfddfd5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:38.186120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:38.192734 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 188/367] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 22 21:56:38 2022] GET /identity/v3/role_assignments?group.id=ec1800a99f56492687b58a547023a1c3&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:38.193633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b52a0-54b0-4b54-aa94-f34e6dfddfd5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:38.222512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a0c62-8a7a-429e-b86e-90be3deca078 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:38.242758 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:38.286591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f823ff0-cad1-4def-acc6-839a3496e47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:38.287218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f823ff0-cad1-4def-acc6-839a3496e47d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:38.287495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f823ff0-cad1-4def-acc6-839a3496e47d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:38.287764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f823ff0-cad1-4def-acc6-839a3496e47d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:38.288488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f823ff0-cad1-4def-acc6-839a3496e47d None admin] RBAC: Adding query filter params (role.id=1f60cad77a004587a328e9dff0cae088, user.id=08c4ee06f486499081496f150e44975e, scope.system=all) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:38.288793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f823ff0-cad1-4def-acc6-839a3496e47d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:38.290276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f823ff0-cad1-4def-acc6-839a3496e47d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:38.308828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 180/368] 10.222.0.25 () {56 vars in 1308 bytes} [Thu Sep 22 21:56:38 2022] GET /identity/v3/role_assignments?user.id=08c4ee06f486499081496f150e44975e&scope.system=all&role.id=1f60cad77a004587a328e9dff0cae088 => generated 510 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:38.317042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f823ff0-cad1-4def-acc6-839a3496e47d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:38.363855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695b52a0-54b0-4b54-aa94-f34e6dfddfd5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:38.417042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97681a7-fe7d-49e2-acff-e4cca5339c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:38.417742 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97681a7-fe7d-49e2-acff-e4cca5339c57 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:38.418019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97681a7-fe7d-49e2-acff-e4cca5339c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:38.418356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97681a7-fe7d-49e2-acff-e4cca5339c57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:38.441552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97681a7-fe7d-49e2-acff-e4cca5339c57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c97681a7-fe7d-49e2-acff-e4cca5339c57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:38.447145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:38.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:38.448898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 189/369] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:38.459326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97681a7-fe7d-49e2-acff-e4cca5339c57 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:38.556423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f823ff0-cad1-4def-acc6-839a3496e47d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:38.585985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:38.626950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cff3c-ed69-4782-a232-3a52dd48cadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:38.628072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cff3c-ed69-4782-a232-3a52dd48cadd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:38.628559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cff3c-ed69-4782-a232-3a52dd48cadd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:38.629078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807cff3c-ed69-4782-a232-3a52dd48cadd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:38.643688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807cff3c-ed69-4782-a232-3a52dd48cadd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:38.648955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97681a7-fe7d-49e2-acff-e4cca5339c57 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-807cff3c-ed69-4782-a232-3a52dd48cadd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:38.649672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:38.650536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:38.651310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:38.656416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 181/370] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:56:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:38.687101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cff3c-ed69-4782-a232-3a52dd48cadd None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:38.701366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e26f1c-8fce-4bf2-93a1-3645990e256e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:38.701366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e26f1c-8fce-4bf2-93a1-3645990e256e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:38.701366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e26f1c-8fce-4bf2-93a1-3645990e256e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:38.701366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e26f1c-8fce-4bf2-93a1-3645990e256e None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:38.701908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e26f1c-8fce-4bf2-93a1-3645990e256e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:38.702161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e26f1c-8fce-4bf2-93a1-3645990e256e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:38.703513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e26f1c-8fce-4bf2-93a1-3645990e256e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:38.717162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 190/371] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:38.720888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e26f1c-8fce-4bf2-93a1-3645990e256e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:38.735798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:38.877331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807cff3c-ed69-4782-a232-3a52dd48cadd None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:38.930429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e26f1c-8fce-4bf2-93a1-3645990e256e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:38.947890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c4e01-b241-448b-9763-c221753a953e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:38.948867 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c4e01-b241-448b-9763-c221753a953e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:38.949364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c4e01-b241-448b-9763-c221753a953e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:38.951453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82c4e01-b241-448b-9763-c221753a953e None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:38.952309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c4e01-b241-448b-9763-c221753a953e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:38.952802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c4e01-b241-448b-9763-c221753a953e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:38.955645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82c4e01-b241-448b-9763-c221753a953e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:38.972627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 182/372] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:38.978543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:38.978685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c4e01-b241-448b-9763-c221753a953e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:38.991047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba3eb1-ff45-4698-bb82-b2309a4cb29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:38.991818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba3eb1-ff45-4698-bb82-b2309a4cb29f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:38.992099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba3eb1-ff45-4698-bb82-b2309a4cb29f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:38.992375 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba3eb1-ff45-4698-bb82-b2309a4cb29f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:39.089051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba3eb1-ff45-4698-bb82-b2309a4cb29f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:39.090751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba3eb1-ff45-4698-bb82-b2309a4cb29f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:39.194622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:39.208646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 191/373] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:39.215229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba3eb1-ff45-4698-bb82-b2309a4cb29f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:39.220125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82c4e01-b241-448b-9763-c221753a953e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:39.241517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:39.321884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db241e9e-74d7-429e-b82b-94a0928cdffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:39.322511 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db241e9e-74d7-429e-b82b-94a0928cdffb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:39.322794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db241e9e-74d7-429e-b82b-94a0928cdffb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:39.323065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db241e9e-74d7-429e-b82b-94a0928cdffb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:39.332183 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba3eb1-ff45-4698-bb82-b2309a4cb29f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:39.370841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25daf7b-0dbb-4293-b487-7143e5d7a395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:39.371997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25daf7b-0dbb-4293-b487-7143e5d7a395 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:39.372472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25daf7b-0dbb-4293-b487-7143e5d7a395 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:39.373225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25daf7b-0dbb-4293-b487-7143e5d7a395 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:39.374669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25daf7b-0dbb-4293-b487-7143e5d7a395 None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:39.376377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25daf7b-0dbb-4293-b487-7143e5d7a395 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:39.378645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 192/374] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:39 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:39.383728 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25daf7b-0dbb-4293-b487-7143e5d7a395 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:39.486807 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db241e9e-74d7-429e-b82b-94a0928cdffb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:39.488880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db241e9e-74d7-429e-b82b-94a0928cdffb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:39.609479 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:39.647934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25daf7b-0dbb-4293-b487-7143e5d7a395 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:39.682794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:39.698111 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eb3c7-5fe5-441b-aa18-2d5655de1b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:39.698756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eb3c7-5fe5-441b-aa18-2d5655de1b1f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:39.699033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eb3c7-5fe5-441b-aa18-2d5655de1b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:39.699304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3eb3c7-5fe5-441b-aa18-2d5655de1b1f None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:39.701224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3eb3c7-5fe5-441b-aa18-2d5655de1b1f None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:39.702560 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 183/375] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 724 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:39.703292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3eb3c7-5fe5-441b-aa18-2d5655de1b1f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:39.709561 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 193/376] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:39 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:39.715528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eb3c7-5fe5-441b-aa18-2d5655de1b1f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:39.715730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db241e9e-74d7-429e-b82b-94a0928cdffb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:39.829780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3eb3c7-5fe5-441b-aa18-2d5655de1b1f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:39.885579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f6068-9755-4942-9bb6-e6c984f14db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:39.886329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f6068-9755-4942-9bb6-e6c984f14db6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:39.886609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f6068-9755-4942-9bb6-e6c984f14db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:39.886876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f6068-9755-4942-9bb6-e6c984f14db6 None admin] PATH_INFO: `/v3/projects/f680186a14a640efb3f020f909ac5fa9` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:39.890721 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f6068-9755-4942-9bb6-e6c984f14db6 None admin] RBAC: Authorizing `identity:get_project(project_id=f680186a14a640efb3f020f909ac5fa9)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:39.893273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f6068-9755-4942-9bb6-e6c984f14db6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:39.899533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 194/377] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:39 2022] GET /identity/v3/projects/f680186a14a640efb3f020f909ac5fa9 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:39.905333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f6068-9755-4942-9bb6-e6c984f14db6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:39.938745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db241e9e-74d7-429e-b82b-94a0928cdffb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:39.956244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:40.021885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67317564-fde7-43de-912c-5f49142342c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:40.022665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317564-fde7-43de-912c-5f49142342c3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:40.023046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317564-fde7-43de-912c-5f49142342c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:40.023418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317564-fde7-43de-912c-5f49142342c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:40.024255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67317564-fde7-43de-912c-5f49142342c3 None admin] RBAC: Adding query filter params (group.id=ec1800a99f56492687b58a547023a1c3, role.id=dea086603b04451680a14ae806205d79, scope.project.id=a36c670998e84f8ba575f12eebf55432) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:40.024686 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67317564-fde7-43de-912c-5f49142342c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:40.026261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67317564-fde7-43de-912c-5f49142342c3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:40.033581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 184/378] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 22 21:56:39 2022] GET /identity/v3/role_assignments?group.id=ec1800a99f56492687b58a547023a1c3&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=dea086603b04451680a14ae806205d79 => generated 260 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:40.076829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f6068-9755-4942-9bb6-e6c984f14db6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:40.131896 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd185991-0690-4604-aad6-4d39de686ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:40.133339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd185991-0690-4604-aad6-4d39de686ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:40.133878 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd185991-0690-4604-aad6-4d39de686ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:40.134305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd185991-0690-4604-aad6-4d39de686ee7 None admin] PATH_INFO: `/v3/groups/6704eea971884cad90ddb3b8834405ab` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:40.142073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd185991-0690-4604-aad6-4d39de686ee7 None admin] RBAC: Authorizing `identity:get_group(group_id=6704eea971884cad90ddb3b8834405ab)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:40.143853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd185991-0690-4604-aad6-4d39de686ee7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:40.146165 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 195/379] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:39 2022] GET /identity/v3/groups/6704eea971884cad90ddb3b8834405ab => generated 229 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:40.153250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67317564-fde7-43de-912c-5f49142342c3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:40.264832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:40.324272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67317564-fde7-43de-912c-5f49142342c3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:40.376622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8489e2-f620-4c16-a633-d2544a4db299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:40.377265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8489e2-f620-4c16-a633-d2544a4db299 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:40.377547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8489e2-f620-4c16-a633-d2544a4db299 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:40.377826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8489e2-f620-4c16-a633-d2544a4db299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:40.378558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8489e2-f620-4c16-a633-d2544a4db299 None admin] RBAC: Adding query filter params (group.id=6704eea971884cad90ddb3b8834405ab, role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=f680186a14a640efb3f020f909ac5fa9) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:40.378866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8489e2-f620-4c16-a633-d2544a4db299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:40.380321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8489e2-f620-4c16-a633-d2544a4db299 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:40.386777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 185/380] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 22 21:56:40 2022] GET /identity/v3/role_assignments?group.id=6704eea971884cad90ddb3b8834405ab&scope.project.id=f680186a14a640efb3f020f909ac5fa9&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:40.632885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44c64-0f70-4c2a-a88d-007fab6e2ad6 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:40.633622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44c64-0f70-4c2a-a88d-007fab6e2ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:40.634119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a44c64-0f70-4c2a-a88d-007fab6e2ad6 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:40.636089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 196/381] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:40.642722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447eaf30-752e-46a0-bcb9-1a048f76bcd6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:40.642974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447eaf30-752e-46a0-bcb9-1a048f76bcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:40.643207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447eaf30-752e-46a0-bcb9-1a048f76bcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:40.670278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-447eaf30-752e-46a0-bcb9-1a048f76bcd6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:40.671545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-447eaf30-752e-46a0-bcb9-1a048f76bcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:40.757205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 186/382] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:41.197135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149311-82b6-4922-9d5b-0c84b1cb4090 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:41.197660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149311-82b6-4922-9d5b-0c84b1cb4090 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:41.197660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca149311-82b6-4922-9d5b-0c84b1cb4090 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:41.199272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 197/383] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:41.205578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc3195-42d4-4827-b90c-b8531afc6815 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:41.206031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc3195-42d4-4827-b90c-b8531afc6815 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:41.206450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc3195-42d4-4827-b90c-b8531afc6815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:41.235552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8bbc3195-42d4-4827-b90c-b8531afc6815 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:41.237021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca149311-82b6-4922-9d5b-0c84b1cb4090 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:41.238882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8bbc3195-42d4-4827-b90c-b8531afc6815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:41.299511 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:41.333297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 187/384] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:41.334509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899ffea-270e-418c-9db4-12e48d1c329a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:41.334796 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899ffea-270e-418c-9db4-12e48d1c329a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:41.335072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9899ffea-270e-418c-9db4-12e48d1c329a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:41.336752 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 188/385] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:41.342320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e35d14-fb2e-4256-af66-1e53ff735f09 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:41.344864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e35d14-fb2e-4256-af66-1e53ff735f09 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:41.346533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e35d14-fb2e-4256-af66-1e53ff735f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:41.367681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d2e35d14-fb2e-4256-af66-1e53ff735f09 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:41.368872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d2e35d14-fb2e-4256-af66-1e53ff735f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:41.380850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca149311-82b6-4922-9d5b-0c84b1cb4090 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:41.393332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:41.431445 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96cd66-900a-4805-9118-b807fd0eec55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:41.432137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96cd66-900a-4805-9118-b807fd0eec55 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:41.432680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96cd66-900a-4805-9118-b807fd0eec55 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:41.432964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a96cd66-900a-4805-9118-b807fd0eec55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:41.447468 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a96cd66-900a-4805-9118-b807fd0eec55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1a96cd66-900a-4805-9118-b807fd0eec55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:41.452814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:41.453772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:41.454637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:41.458856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 198/386] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:41.459184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 189/387] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:41.462696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96cd66-900a-4805-9118-b807fd0eec55 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:41.569594 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a96cd66-900a-4805-9118-b807fd0eec55 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:41.608233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070523f-e8aa-4bf1-a023-bca2cbf244fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:41.608767 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070523f-e8aa-4bf1-a023-bca2cbf244fe None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:41.609045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070523f-e8aa-4bf1-a023-bca2cbf244fe None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:41.609343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070523f-e8aa-4bf1-a023-bca2cbf244fe None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:41.610080 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070523f-e8aa-4bf1-a023-bca2cbf244fe None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:41.610383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070523f-e8aa-4bf1-a023-bca2cbf244fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:41.611859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070523f-e8aa-4bf1-a023-bca2cbf244fe None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:41.624478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 199/388] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:41.630862 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e35d14-fb2e-4256-af66-1e53ff735f09 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:41.726187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070523f-e8aa-4bf1-a023-bca2cbf244fe None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:41.750436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:41.769926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e35d14-fb2e-4256-af66-1e53ff735f09 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:41.789824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:41.835276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b91175-9b8b-4434-b00a-29c22f7b3710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:41.836389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b91175-9b8b-4434-b00a-29c22f7b3710 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:41.836515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b91175-9b8b-4434-b00a-29c22f7b3710 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:41.836878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b91175-9b8b-4434-b00a-29c22f7b3710 None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:41.838158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b91175-9b8b-4434-b00a-29c22f7b3710 None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:41.839966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b91175-9b8b-4434-b00a-29c22f7b3710 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:41.844901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 190/389] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:41 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:41.846048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b91175-9b8b-4434-b00a-29c22f7b3710 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:41.887160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070523f-e8aa-4bf1-a023-bca2cbf244fe None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:41.927890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ee6e9-af2d-4129-a5cb-787e41b157f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:41.928505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ee6e9-af2d-4129-a5cb-787e41b157f2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:41.928785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ee6e9-af2d-4129-a5cb-787e41b157f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:41.929056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36ee6e9-af2d-4129-a5cb-787e41b157f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:41.941550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36ee6e9-af2d-4129-a5cb-787e41b157f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a36ee6e9-af2d-4129-a5cb-787e41b157f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:41.946890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:41.947759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:41.948533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 200/390] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:41.950322 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ee6e9-af2d-4129-a5cb-787e41b157f2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:41.977939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b91175-9b8b-4434-b00a-29c22f7b3710 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:42.025769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6177035-b31b-4142-a7b5-ccb07a07e671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:42.026754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6177035-b31b-4142-a7b5-ccb07a07e671 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.027188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6177035-b31b-4142-a7b5-ccb07a07e671 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.027601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6177035-b31b-4142-a7b5-ccb07a07e671 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.036205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:42.041366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6177035-b31b-4142-a7b5-ccb07a07e671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f6177035-b31b-4142-a7b5-ccb07a07e671 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:42.049933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:42.050995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:42.052024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:42.054725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 191/391] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:42.055933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6177035-b31b-4142-a7b5-ccb07a07e671 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:42.075716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36ee6e9-af2d-4129-a5cb-787e41b157f2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:42.107294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:42.120324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e72906-1242-4c71-8b6c-4253e64dc91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:42.122130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e72906-1242-4c71-8b6c-4253e64dc91a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.122130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e72906-1242-4c71-8b6c-4253e64dc91a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.123049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e72906-1242-4c71-8b6c-4253e64dc91a None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.124465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e72906-1242-4c71-8b6c-4253e64dc91a None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:42.126023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e72906-1242-4c71-8b6c-4253e64dc91a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:42.128354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 201/392] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:41 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:42.129342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e72906-1242-4c71-8b6c-4253e64dc91a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:42.178199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6177035-b31b-4142-a7b5-ccb07a07e671 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:42.217151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b397d-e5e5-4a23-bdf5-7b55eca133f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:42.218630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b397d-e5e5-4a23-bdf5-7b55eca133f5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.219079 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b397d-e5e5-4a23-bdf5-7b55eca133f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.219515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b397d-e5e5-4a23-bdf5-7b55eca133f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.220388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b397d-e5e5-4a23-bdf5-7b55eca133f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:42.220884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b397d-e5e5-4a23-bdf5-7b55eca133f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:42.222662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52b397d-e5e5-4a23-bdf5-7b55eca133f5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:42.232574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 192/393] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:42.233703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b397d-e5e5-4a23-bdf5-7b55eca133f5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:42.244141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e72906-1242-4c71-8b6c-4253e64dc91a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:42.280818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f64db3-5d22-4c92-a78d-92ac6cc88b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:42.282122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f64db3-5d22-4c92-a78d-92ac6cc88b55 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.282487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f64db3-5d22-4c92-a78d-92ac6cc88b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.282822 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f64db3-5d22-4c92-a78d-92ac6cc88b55 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.283732 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f64db3-5d22-4c92-a78d-92ac6cc88b55 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:42.284010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f64db3-5d22-4c92-a78d-92ac6cc88b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:42.285490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f64db3-5d22-4c92-a78d-92ac6cc88b55 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:42.295800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 202/394] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:42.296725 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f64db3-5d22-4c92-a78d-92ac6cc88b55 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:42.301522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:42.347804 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:42.350877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52b397d-e5e5-4a23-bdf5-7b55eca133f5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:42.385888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21495abe-81bb-463e-9011-6c398bc008c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:42.386950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21495abe-81bb-463e-9011-6c398bc008c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.387455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21495abe-81bb-463e-9011-6c398bc008c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.387914 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21495abe-81bb-463e-9011-6c398bc008c4 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450/groups/ec1800a99f56492687b58a547023a1c3/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.390029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21495abe-81bb-463e-9011-6c398bc008c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8087b41e58fb4aca97cbfced2c011450, group_id=ec1800a99f56492687b58a547023a1c3, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:42.391691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21495abe-81bb-463e-9011-6c398bc008c4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:42.409173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f64db3-5d22-4c92-a78d-92ac6cc88b55 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:42.409728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 193/395] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 22 21:56:42 2022] PUT /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450/groups/ec1800a99f56492687b58a547023a1c3/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:42.411058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21495abe-81bb-463e-9011-6c398bc008c4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:42.478314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9bfc8-c0e1-4d12-b8f6-5f2031ef6ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:42.479037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bfc8-c0e1-4d12-b8f6-5f2031ef6ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.479316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bfc8-c0e1-4d12-b8f6-5f2031ef6ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.479588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9bfc8-c0e1-4d12-b8f6-5f2031ef6ea7 None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.482718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9bfc8-c0e1-4d12-b8f6-5f2031ef6ea7 None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:42.484307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9bfc8-c0e1-4d12-b8f6-5f2031ef6ea7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:42.486392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 203/396] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:42 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:42.493049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9bfc8-c0e1-4d12-b8f6-5f2031ef6ea7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:42.534171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21495abe-81bb-463e-9011-6c398bc008c4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:42.575317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54147e8-de4c-495f-99b4-f214dabac4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:42.577227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54147e8-de4c-495f-99b4-f214dabac4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.577720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54147e8-de4c-495f-99b4-f214dabac4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.578192 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54147e8-de4c-495f-99b4-f214dabac4e3 None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.579554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54147e8-de4c-495f-99b4-f214dabac4e3 None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:42.581654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54147e8-de4c-495f-99b4-f214dabac4e3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:42.583492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 194/397] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:42 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:42.589449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54147e8-de4c-495f-99b4-f214dabac4e3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:42.623654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:42.675058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:42.716442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54147e8-de4c-495f-99b4-f214dabac4e3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:42.720526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9bfc8-c0e1-4d12-b8f6-5f2031ef6ea7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:42.767944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db37ee1-cd27-42e8-bb26-54af711b3835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:42.768588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db37ee1-cd27-42e8-bb26-54af711b3835 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.768863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db37ee1-cd27-42e8-bb26-54af711b3835 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.769141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db37ee1-cd27-42e8-bb26-54af711b3835 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.771343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db37ee1-cd27-42e8-bb26-54af711b3835 None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:42.773849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db37ee1-cd27-42e8-bb26-54af711b3835 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:42.775647 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 195/398] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:42 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:42.777573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bea9be-fe4d-46bb-928c-3459b28d451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:42.778952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bea9be-fe4d-46bb-928c-3459b28d451e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.779134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bea9be-fe4d-46bb-928c-3459b28d451e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.779405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bea9be-fe4d-46bb-928c-3459b28d451e None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.780745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bea9be-fe4d-46bb-928c-3459b28d451e None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:42.782058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bea9be-fe4d-46bb-928c-3459b28d451e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:42.783334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db37ee1-cd27-42e8-bb26-54af711b3835 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:42.784889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 204/399] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:42 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:42.796690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bea9be-fe4d-46bb-928c-3459b28d451e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:42.912609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bea9be-fe4d-46bb-928c-3459b28d451e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:42.918171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:42.926544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db37ee1-cd27-42e8-bb26-54af711b3835 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:42.956401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c05d655-ce66-4a29-b8ea-a9e87c2f4d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:42.957685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05d655-ce66-4a29-b8ea-a9e87c2f4d0e None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.958347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05d655-ce66-4a29-b8ea-a9e87c2f4d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.958772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c05d655-ce66-4a29-b8ea-a9e87c2f4d0e None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450/groups/ec1800a99f56492687b58a547023a1c3/roles/dea086603b04451680a14ae806205d79` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.961789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05d655-ce66-4a29-b8ea-a9e87c2f4d0e None admin] RBAC: Authorizing `identity:create_grant(project_id=8087b41e58fb4aca97cbfced2c011450, group_id=ec1800a99f56492687b58a547023a1c3, role_id=dea086603b04451680a14ae806205d79)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:42.964096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c05d655-ce66-4a29-b8ea-a9e87c2f4d0e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:42.969810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3f046-ed32-4140-9dba-5fa5ae9c7c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:42.970635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3f046-ed32-4140-9dba-5fa5ae9c7c8a None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.970925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3f046-ed32-4140-9dba-5fa5ae9c7c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.971200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3f046-ed32-4140-9dba-5fa5ae9c7c8a None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432/groups/ec1800a99f56492687b58a547023a1c3/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.973401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3f046-ed32-4140-9dba-5fa5ae9c7c8a None admin] RBAC: Authorizing `identity:create_grant(project_id=a36c670998e84f8ba575f12eebf55432, group_id=ec1800a99f56492687b58a547023a1c3, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:42.975035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3f046-ed32-4140-9dba-5fa5ae9c7c8a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:42.975851 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:42.983052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 205/400] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 22 21:56:42 2022] PUT /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450/groups/ec1800a99f56492687b58a547023a1c3/roles/dea086603b04451680a14ae806205d79 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 21:56:42.984696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c764af-0274-431b-94a4-b31c5b783ea5 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.985220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c764af-0274-431b-94a4-b31c5b783ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.985692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c764af-0274-431b-94a4-b31c5b783ea5 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:42.987601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 206/401] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:42.993366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e444f0-69f7-4105-a7fd-1452b74722c5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:42.994013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e444f0-69f7-4105-a7fd-1452b74722c5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:42.994477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e444f0-69f7-4105-a7fd-1452b74722c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:43.001301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 196/402] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 22 21:56:42 2022] PUT /identity/v3/projects/a36c670998e84f8ba575f12eebf55432/groups/ec1800a99f56492687b58a547023a1c3/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:43.016466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-08e444f0-69f7-4105-a7fd-1452b74722c5 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:43.018141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-08e444f0-69f7-4105-a7fd-1452b74722c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:43.108393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 207/403] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:43.377925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3f046-ed32-4140-9dba-5fa5ae9c7c8a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:43.521457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3f046-ed32-4140-9dba-5fa5ae9c7c8a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:43.535163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6fb65-9c84-4bbc-9fdc-3459cc2d1f11 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:43.535676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6fb65-9c84-4bbc-9fdc-3459cc2d1f11 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:43.535676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c6fb65-9c84-4bbc-9fdc-3459cc2d1f11 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:43.537005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 208/404] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:43.541965 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eca98-fd71-4452-a1f0-450550168d06 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:43.542249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eca98-fd71-4452-a1f0-450550168d06 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:43.542517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222eca98-fd71-4452-a1f0-450550168d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:43.570127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-222eca98-fd71-4452-a1f0-450550168d06 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:43.571310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-222eca98-fd71-4452-a1f0-450550168d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:43.585630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8039b-0f78-411a-83a2-1076ce4e996e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:43.586253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8039b-0f78-411a-83a2-1076ce4e996e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:43.586956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8039b-0f78-411a-83a2-1076ce4e996e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:43.587236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a8039b-0f78-411a-83a2-1076ce4e996e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:43.598885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a8039b-0f78-411a-83a2-1076ce4e996e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-d2a8039b-0f78-411a-83a2-1076ce4e996e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:43.604629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:43.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:43.607129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 197/405] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:43.612915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8039b-0f78-411a-83a2-1076ce4e996e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:43.638269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:43.729059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a8039b-0f78-411a-83a2-1076ce4e996e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:43.767285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a8fc4-5737-461d-9203-8d02d961ab22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:43.767882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a8fc4-5737-461d-9203-8d02d961ab22 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:43.768154 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a8fc4-5737-461d-9203-8d02d961ab22 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:43.768418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a8fc4-5737-461d-9203-8d02d961ab22 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:43.769058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a8fc4-5737-461d-9203-8d02d961ab22 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:43.769381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a8fc4-5737-461d-9203-8d02d961ab22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:43.771068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a8fc4-5737-461d-9203-8d02d961ab22 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:43.778430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 209/406] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:43.781550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 198/407] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:43.786998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222eca98-fd71-4452-a1f0-450550168d06 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:43.848584 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:43.890589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222eca98-fd71-4452-a1f0-450550168d06 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:43.926687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b8d12-b834-48eb-a90b-69353fb7ca45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:43.927318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b8d12-b834-48eb-a90b-69353fb7ca45 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:43.927591 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b8d12-b834-48eb-a90b-69353fb7ca45 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:43.927858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b8d12-b834-48eb-a90b-69353fb7ca45 None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:43.928953 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b8d12-b834-48eb-a90b-69353fb7ca45 None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:43.930413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b8d12-b834-48eb-a90b-69353fb7ca45 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:43.932317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 210/408] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:43 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:43.937830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a8fc4-5737-461d-9203-8d02d961ab22 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:44.019639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:44.046886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a8fc4-5737-461d-9203-8d02d961ab22 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:44.102916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef4146-2fd4-4beb-940e-ace58092ab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:44.103541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef4146-2fd4-4beb-940e-ace58092ab12 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:44.103817 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef4146-2fd4-4beb-940e-ace58092ab12 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:44.104084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ef4146-2fd4-4beb-940e-ace58092ab12 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:44.106498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef4146-2fd4-4beb-940e-ace58092ab12 None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:44.107947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ef4146-2fd4-4beb-940e-ace58092ab12 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:44.110589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 199/409] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:43 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:44.115332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b8d12-b834-48eb-a90b-69353fb7ca45 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:44.165959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef4146-2fd4-4beb-940e-ace58092ab12 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:44.280512 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b8d12-b834-48eb-a90b-69353fb7ca45 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:44.347554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ef4146-2fd4-4beb-940e-ace58092ab12 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:44.354603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:44.383960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddc076-6d4b-434f-8982-3190a0d46f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:44.384598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc076-6d4b-434f-8982-3190a0d46f40 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:44.384849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc076-6d4b-434f-8982-3190a0d46f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:44.385130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc076-6d4b-434f-8982-3190a0d46f40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:44.396152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddc076-6d4b-434f-8982-3190a0d46f40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-b6ddc076-6d4b-434f-8982-3190a0d46f40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:44.401624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:44.402530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:44.403995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 200/410] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:56:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:44.408843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9c9fb-3bcd-42ce-8573-53b8b715acc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:44.409551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9c9fb-3bcd-42ce-8573-53b8b715acc5 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:44.409830 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9c9fb-3bcd-42ce-8573-53b8b715acc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:44.410105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9c9fb-3bcd-42ce-8573-53b8b715acc5 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432/groups/ec1800a99f56492687b58a547023a1c3/roles/dea086603b04451680a14ae806205d79` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:44.410728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddc076-6d4b-434f-8982-3190a0d46f40 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:44.420520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:44.421888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9c9fb-3bcd-42ce-8573-53b8b715acc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=a36c670998e84f8ba575f12eebf55432, group_id=ec1800a99f56492687b58a547023a1c3, role_id=dea086603b04451680a14ae806205d79)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:44.423542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9c9fb-3bcd-42ce-8573-53b8b715acc5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:44.439154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 211/411] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 22 21:56:44 2022] PUT /identity/v3/projects/a36c670998e84f8ba575f12eebf55432/groups/ec1800a99f56492687b58a547023a1c3/roles/dea086603b04451680a14ae806205d79 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:44.578214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddc076-6d4b-434f-8982-3190a0d46f40 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:44.624210 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d638209-1862-430a-bb12-fb7a75c47814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:44.625146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d638209-1862-430a-bb12-fb7a75c47814 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:44.625630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d638209-1862-430a-bb12-fb7a75c47814 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:44.626105 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d638209-1862-430a-bb12-fb7a75c47814 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:44.626954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d638209-1862-430a-bb12-fb7a75c47814 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:44.627434 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d638209-1862-430a-bb12-fb7a75c47814 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:44.628981 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d638209-1862-430a-bb12-fb7a75c47814 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:44.639027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 201/412] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:44.644260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9c9fb-3bcd-42ce-8573-53b8b715acc5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:44.758326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9c9fb-3bcd-42ce-8573-53b8b715acc5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:44.796506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eb6a1-26b3-4d74-bd55-6a7d66ed1cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:44.797453 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eb6a1-26b3-4d74-bd55-6a7d66ed1cec None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:44.797924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eb6a1-26b3-4d74-bd55-6a7d66ed1cec None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:44.798388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484eb6a1-26b3-4d74-bd55-6a7d66ed1cec None admin] PATH_INFO: `/v3/groups/6704eea971884cad90ddb3b8834405ab` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:44.799682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eb6a1-26b3-4d74-bd55-6a7d66ed1cec None admin] RBAC: Authorizing `identity:get_group(group_id=6704eea971884cad90ddb3b8834405ab)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:44.801313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484eb6a1-26b3-4d74-bd55-6a7d66ed1cec None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:44.803058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 212/413] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:44 2022] GET /identity/v3/groups/6704eea971884cad90ddb3b8834405ab => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:44.810622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d638209-1862-430a-bb12-fb7a75c47814 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:44.909629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:44.945706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d638209-1862-430a-bb12-fb7a75c47814 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:44.980356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b40689-03e5-4120-bd86-a0cda0366127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:44.981662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b40689-03e5-4120-bd86-a0cda0366127 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:44.981943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b40689-03e5-4120-bd86-a0cda0366127 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:44.982406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b40689-03e5-4120-bd86-a0cda0366127 None admin] PATH_INFO: `/v3/projects/f680186a14a640efb3f020f909ac5fa9` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:44.983546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b40689-03e5-4120-bd86-a0cda0366127 None admin] RBAC: Authorizing `identity:get_project(project_id=f680186a14a640efb3f020f909ac5fa9)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:44.985026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b40689-03e5-4120-bd86-a0cda0366127 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:44.987130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 202/414] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:44 2022] GET /identity/v3/projects/f680186a14a640efb3f020f909ac5fa9 => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:44.994809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eb6a1-26b3-4d74-bd55-6a7d66ed1cec None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:45.095500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:45.122914 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22add464-cf68-4655-8316-0d7ef942f353 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:45.123258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22add464-cf68-4655-8316-0d7ef942f353 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:45.123535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22add464-cf68-4655-8316-0d7ef942f353 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:45.126392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 203/415] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:45.138691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f310a-f50c-46ee-816d-7e1c530a14dc None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:45.138989 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f310a-f50c-46ee-816d-7e1c530a14dc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:45.139236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20f310a-f50c-46ee-816d-7e1c530a14dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:45.160271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a20f310a-f50c-46ee-816d-7e1c530a14dc None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:45.161757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a20f310a-f50c-46ee-816d-7e1c530a14dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:45.168969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484eb6a1-26b3-4d74-bd55-6a7d66ed1cec None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:45.244744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19277c83-c179-4c77-b7d5-d51356f1fdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:45.245431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19277c83-c179-4c77-b7d5-d51356f1fdc2 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:45.245702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19277c83-c179-4c77-b7d5-d51356f1fdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:45.245969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19277c83-c179-4c77-b7d5-d51356f1fdc2 None admin] PATH_INFO: `/v3/projects/f680186a14a640efb3f020f909ac5fa9/groups/6704eea971884cad90ddb3b8834405ab/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:45.249776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19277c83-c179-4c77-b7d5-d51356f1fdc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f680186a14a640efb3f020f909ac5fa9, group_id=6704eea971884cad90ddb3b8834405ab, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:45.251195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19277c83-c179-4c77-b7d5-d51356f1fdc2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:45.273231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 213/416] 10.222.0.25 () {58 vars in 1442 bytes} [Thu Sep 22 21:56:44 2022] PUT /identity/v3/projects/f680186a14a640efb3f020f909ac5fa9/groups/6704eea971884cad90ddb3b8834405ab/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:56:45.299915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 204/417] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:45.720423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19277c83-c179-4c77-b7d5-d51356f1fdc2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:45.775383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3e84-f969-4765-b3a5-3885a8885584 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:45.775924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3e84-f969-4765-b3a5-3885a8885584 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:45.775924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3e84-f969-4765-b3a5-3885a8885584 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:45.777549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 205/418] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:45.783567 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aedb43-90eb-48d2-b94e-e9e9bc1c97ad None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:45.783849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aedb43-90eb-48d2-b94e-e9e9bc1c97ad None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:45.784124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aedb43-90eb-48d2-b94e-e9e9bc1c97ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:45.804454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-81aedb43-90eb-48d2-b94e-e9e9bc1c97ad None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:45.806286 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-81aedb43-90eb-48d2-b94e-e9e9bc1c97ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:45.851298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19277c83-c179-4c77-b7d5-d51356f1fdc2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:45.869125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:45.889081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:45.896773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612787d-9fb9-4c51-8049-60f22a5dd5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:45.898870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612787d-9fb9-4c51-8049-60f22a5dd5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:45.898870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612787d-9fb9-4c51-8049-60f22a5dd5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:45.898870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0612787d-9fb9-4c51-8049-60f22a5dd5f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:45.899297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 206/419] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:45.901705 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b63cd-b8ae-4b71-896b-693f12470324 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:45.901981 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b63cd-b8ae-4b71-896b-693f12470324 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:45.902248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b63cd-b8ae-4b71-896b-693f12470324 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:45.903803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 207/420] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:45.912091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2690e-31ea-4023-adef-3bf96544191d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:45.912367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2690e-31ea-4023-adef-3bf96544191d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:45.912630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2690e-31ea-4023-adef-3bf96544191d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:45.923834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0612787d-9fb9-4c51-8049-60f22a5dd5f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-0612787d-9fb9-4c51-8049-60f22a5dd5f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:45.937457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:45.938454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:45.939636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 214/421] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:45.941854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-83e2690e-31ea-4023-adef-3bf96544191d None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:45.943039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-83e2690e-31ea-4023-adef-3bf96544191d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:45.944462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612787d-9fb9-4c51-8049-60f22a5dd5f2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:46.070581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 208/422] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:46.112035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0612787d-9fb9-4c51-8049-60f22a5dd5f2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:46.146288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e70c49-439a-4c8c-9a6c-a71893645cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:46.146897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e70c49-439a-4c8c-9a6c-a71893645cbc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:46.147176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e70c49-439a-4c8c-9a6c-a71893645cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:46.147449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e70c49-439a-4c8c-9a6c-a71893645cbc None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:46.148110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e70c49-439a-4c8c-9a6c-a71893645cbc None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:46.148414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e70c49-439a-4c8c-9a6c-a71893645cbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:46.149797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e70c49-439a-4c8c-9a6c-a71893645cbc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:46.158799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 215/423] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:46.166199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2690e-31ea-4023-adef-3bf96544191d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:46.260744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e70c49-439a-4c8c-9a6c-a71893645cbc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:46.309601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:46.312961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2690e-31ea-4023-adef-3bf96544191d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:46.361472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:46.372716 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b5cc1-9077-4375-b532-7431e2458f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:46.373392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5cc1-9077-4375-b532-7431e2458f5e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:46.373680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5cc1-9077-4375-b532-7431e2458f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:46.373951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38b5cc1-9077-4375-b532-7431e2458f5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:46.384132 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e70c49-439a-4c8c-9a6c-a71893645cbc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:46.421886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fabfe-5edc-45f3-82ec-d33340fa0324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:46.422670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fabfe-5edc-45f3-82ec-d33340fa0324 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:46.423047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fabfe-5edc-45f3-82ec-d33340fa0324 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:46.423415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fabfe-5edc-45f3-82ec-d33340fa0324 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:46.435456 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fabfe-5edc-45f3-82ec-d33340fa0324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-776fabfe-5edc-45f3-82ec-d33340fa0324 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:46.440794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:46.441792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 22 21:56:46.442598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:46.445371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 216/424] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:46.448496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fabfe-5edc-45f3-82ec-d33340fa0324 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:46.495112 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b5cc1-9077-4375-b532-7431e2458f5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:46.497220 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38b5cc1-9077-4375-b532-7431e2458f5e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:46.557780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:46.563987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fabfe-5edc-45f3-82ec-d33340fa0324 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:46.602827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff686c17-20de-4068-b0ef-8b01aeb55367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:46.603541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff686c17-20de-4068-b0ef-8b01aeb55367 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:46.603894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff686c17-20de-4068-b0ef-8b01aeb55367 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:46.604239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff686c17-20de-4068-b0ef-8b01aeb55367 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:46.604969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff686c17-20de-4068-b0ef-8b01aeb55367 None admin] RBAC: Adding query filter params (name=member) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:46.605380 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff686c17-20de-4068-b0ef-8b01aeb55367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:46.607752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff686c17-20de-4068-b0ef-8b01aeb55367 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:46.621264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 217/425] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:56:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:46.623628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff686c17-20de-4068-b0ef-8b01aeb55367 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:46.703086 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 209/426] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:46.704025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b5cc1-9077-4375-b532-7431e2458f5e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:46.731970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:46.739173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff686c17-20de-4068-b0ef-8b01aeb55367 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:46.789454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d9b2a-f7f8-4a5a-a356-a87d9a179ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:46.790471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d9b2a-f7f8-4a5a-a356-a87d9a179ddf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:46.790810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d9b2a-f7f8-4a5a-a356-a87d9a179ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:46.791236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591d9b2a-f7f8-4a5a-a356-a87d9a179ddf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:46.813205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38b5cc1-9077-4375-b532-7431e2458f5e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:46.829009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:46.853671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2524d-aa72-4df1-9c1b-9e155ef08bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:46.854479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2524d-aa72-4df1-9c1b-9e155ef08bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:46.854884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2524d-aa72-4df1-9c1b-9e155ef08bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:46.855346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2524d-aa72-4df1-9c1b-9e155ef08bf6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:46.868771 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2524d-aa72-4df1-9c1b-9e155ef08bf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-42d2524d-aa72-4df1-9c1b-9e155ef08bf6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:46.875005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:46.876127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:46.877176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:46.883595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 210/427] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:46.884500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2524d-aa72-4df1-9c1b-9e155ef08bf6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:46.884846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d9b2a-f7f8-4a5a-a356-a87d9a179ddf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:46.886336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591d9b2a-f7f8-4a5a-a356-a87d9a179ddf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:46.981905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:46.996871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2524d-aa72-4df1-9c1b-9e155ef08bf6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:47.000242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 218/428] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 21:56:47.001280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d9b2a-f7f8-4a5a-a356-a87d9a179ddf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:47.032285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a8100-daa0-4ff2-bb24-3a7c9a5dee8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:47.032904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a8100-daa0-4ff2-bb24-3a7c9a5dee8b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.033203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a8100-daa0-4ff2-bb24-3a7c9a5dee8b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.033476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a8100-daa0-4ff2-bb24-3a7c9a5dee8b None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.034588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a8100-daa0-4ff2-bb24-3a7c9a5dee8b None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:47.036002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a8100-daa0-4ff2-bb24-3a7c9a5dee8b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:47.037788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 211/429] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:46 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:47.038700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c386af-c20d-4e9c-b9d6-f51ea7601e06 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.038976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c386af-c20d-4e9c-b9d6-f51ea7601e06 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.039242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c386af-c20d-4e9c-b9d6-f51ea7601e06 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.040561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 212/430] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:47.045699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c386af-c20d-4e9c-b9d6-f51ea7601e06 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:47.072325 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:47.118786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591d9b2a-f7f8-4a5a-a356-a87d9a179ddf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:47.157195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f692d3-5470-461c-a5c5-6ca5dfc7a919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:47.157793 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f692d3-5470-461c-a5c5-6ca5dfc7a919 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.158132 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f692d3-5470-461c-a5c5-6ca5dfc7a919 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.158340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f692d3-5470-461c-a5c5-6ca5dfc7a919 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.158995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f692d3-5470-461c-a5c5-6ca5dfc7a919 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:47.159227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c386af-c20d-4e9c-b9d6-f51ea7601e06 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:47.159327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f692d3-5470-461c-a5c5-6ca5dfc7a919 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:47.160649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f692d3-5470-461c-a5c5-6ca5dfc7a919 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:47.169748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 219/431] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:47.174957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f692d3-5470-461c-a5c5-6ca5dfc7a919 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:47.194096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97576af-e2be-43e9-9f8d-ff0a684f78bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:47.194742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97576af-e2be-43e9-9f8d-ff0a684f78bc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.195015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97576af-e2be-43e9-9f8d-ff0a684f78bc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.195283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97576af-e2be-43e9-9f8d-ff0a684f78bc None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.196391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97576af-e2be-43e9-9f8d-ff0a684f78bc None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:47.197844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97576af-e2be-43e9-9f8d-ff0a684f78bc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:47.200053 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 213/432] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:47 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:47.201153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85411698-2d2b-43c9-b7c1-ec6d977554af None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.201437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85411698-2d2b-43c9-b7c1-ec6d977554af None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.201707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85411698-2d2b-43c9-b7c1-ec6d977554af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.219709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-85411698-2d2b-43c9-b7c1-ec6d977554af None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:47.220873 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-85411698-2d2b-43c9-b7c1-ec6d977554af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:47.237570 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:47.294230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f692d3-5470-461c-a5c5-6ca5dfc7a919 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:47.296920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 214/433] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:47.297983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85411698-2d2b-43c9-b7c1-ec6d977554af None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:47.330197 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:47.350638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29956aa1-dbb8-4f7d-b1f3-047e89864389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:47.351417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29956aa1-dbb8-4f7d-b1f3-047e89864389 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.352185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29956aa1-dbb8-4f7d-b1f3-047e89864389 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.352566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29956aa1-dbb8-4f7d-b1f3-047e89864389 None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.353843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29956aa1-dbb8-4f7d-b1f3-047e89864389 None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:47.356006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29956aa1-dbb8-4f7d-b1f3-047e89864389 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:47.362699 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 220/434] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:47 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:47.363908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29956aa1-dbb8-4f7d-b1f3-047e89864389 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:47.425734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85411698-2d2b-43c9-b7c1-ec6d977554af None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:47.470530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29956aa1-dbb8-4f7d-b1f3-047e89864389 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:47.475135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c4921-2138-4078-ac90-ccb371c6cb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:47.476001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c4921-2138-4078-ac90-ccb371c6cb22 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.476480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c4921-2138-4078-ac90-ccb371c6cb22 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.476970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c4921-2138-4078-ac90-ccb371c6cb22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.502113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:47.518724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc50f1-c045-4108-ab2a-ae95a8e2e588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:47.519674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc50f1-c045-4108-ab2a-ae95a8e2e588 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.520155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc50f1-c045-4108-ab2a-ae95a8e2e588 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.520623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc50f1-c045-4108-ab2a-ae95a8e2e588 None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.521985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc50f1-c045-4108-ab2a-ae95a8e2e588 None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:47.524141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc50f1-c045-4108-ab2a-ae95a8e2e588 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:47.532233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 221/435] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:47 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:47.532233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc50f1-c045-4108-ab2a-ae95a8e2e588 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:47.559690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c4921-2138-4078-ac90-ccb371c6cb22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:47.561191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c4921-2138-4078-ac90-ccb371c6cb22 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:47.607093 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:47.655246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc50f1-c045-4108-ab2a-ae95a8e2e588 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:47.668648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 215/436] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:47.668648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c4921-2138-4078-ac90-ccb371c6cb22 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:47.696710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe4ac-9314-4458-84e9-559217f76dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:47.697403 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe4ac-9314-4458-84e9-559217f76dfd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.697681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe4ac-9314-4458-84e9-559217f76dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.698036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cfe4ac-9314-4458-84e9-559217f76dfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.698819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe4ac-9314-4458-84e9-559217f76dfd None admin] RBAC: Adding query filter params (group.id=ec1800a99f56492687b58a547023a1c3, role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, scope.project.id=8087b41e58fb4aca97cbfced2c011450) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:47.699150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe4ac-9314-4458-84e9-559217f76dfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:47.700829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cfe4ac-9314-4458-84e9-559217f76dfd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:47.707506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 222/437] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 22 21:56:47 2022] GET /identity/v3/role_assignments?group.id=ec1800a99f56492687b58a547023a1c3&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:47.707506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe4ac-9314-4458-84e9-559217f76dfd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:47.728613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:47.794858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c4921-2138-4078-ac90-ccb371c6cb22 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:47.847098 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfcb77-3e59-4a03-a63a-dfcb945e36f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:47.848120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfcb77-3e59-4a03-a63a-dfcb945e36f7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.848399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfcb77-3e59-4a03-a63a-dfcb945e36f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.848805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decfcb77-3e59-4a03-a63a-dfcb945e36f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.849698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfcb77-3e59-4a03-a63a-dfcb945e36f7 None admin] RBAC: Adding query filter params (group.id=ec1800a99f56492687b58a547023a1c3, role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be, scope.project.id=a36c670998e84f8ba575f12eebf55432) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:47.850003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfcb77-3e59-4a03-a63a-dfcb945e36f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:47.851017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cfe4ac-9314-4458-84e9-559217f76dfd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:47.851451 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decfcb77-3e59-4a03-a63a-dfcb945e36f7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:47.860698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 216/438] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 22 21:56:47 2022] GET /identity/v3/role_assignments?group.id=ec1800a99f56492687b58a547023a1c3&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:47.861601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfcb77-3e59-4a03-a63a-dfcb945e36f7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:47.879162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:47.899770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87208a79-fd6c-4d66-80c5-bf825b2d2a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:47.900582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87208a79-fd6c-4d66-80c5-bf825b2d2a6b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.900960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87208a79-fd6c-4d66-80c5-bf825b2d2a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.901363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87208a79-fd6c-4d66-80c5-bf825b2d2a6b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.915533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87208a79-fd6c-4d66-80c5-bf825b2d2a6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-87208a79-fd6c-4d66-80c5-bf825b2d2a6b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:47.924763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:47.926973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 223/439] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:47.929239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8c44f-4b47-49d2-ac14-968e68bc380a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:47.930973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8c44f-4b47-49d2-ac14-968e68bc380a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:47.930973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8c44f-4b47-49d2-ac14-968e68bc380a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:47.930973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 224/440] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:47.937125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8c44f-4b47-49d2-ac14-968e68bc380a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:48.008350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decfcb77-3e59-4a03-a63a-dfcb945e36f7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:48.030458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:48.060052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4b25f2-3997-4083-bd50-0d8cd56d2033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:48.060348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b25f2-3997-4083-bd50-0d8cd56d2033 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:48.060584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b25f2-3997-4083-bd50-0d8cd56d2033 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:48.060853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4b25f2-3997-4083-bd50-0d8cd56d2033 None admin] PATH_INFO: `/v3/projects/8087b41e58fb4aca97cbfced2c011450` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:48.063886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b25f2-3997-4083-bd50-0d8cd56d2033 None admin] RBAC: Authorizing `identity:get_project(project_id=8087b41e58fb4aca97cbfced2c011450)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:48.064548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4b25f2-3997-4083-bd50-0d8cd56d2033 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:48.066247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 217/441] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:47 2022] GET /identity/v3/projects/8087b41e58fb4aca97cbfced2c011450 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:48.067401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994f886-6eb3-4a1a-b60e-489636b44f46 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:48.067928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994f886-6eb3-4a1a-b60e-489636b44f46 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:48.067928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9994f886-6eb3-4a1a-b60e-489636b44f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:48.080736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8c44f-4b47-49d2-ac14-968e68bc380a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:48.095144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9994f886-6eb3-4a1a-b60e-489636b44f46 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:48.096634 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9994f886-6eb3-4a1a-b60e-489636b44f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:48.121276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb77ba0-638e-4786-bd40-76f84229902d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:48.121652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb77ba0-638e-4786-bd40-76f84229902d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:48.122007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb77ba0-638e-4786-bd40-76f84229902d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:48.122279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb77ba0-638e-4786-bd40-76f84229902d None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:48.122950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb77ba0-638e-4786-bd40-76f84229902d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:48.123247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb77ba0-638e-4786-bd40-76f84229902d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:48.124891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb77ba0-638e-4786-bd40-76f84229902d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:48.139915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 225/442] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:48.140782 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb77ba0-638e-4786-bd40-76f84229902d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:48.179999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 218/443] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:48.180933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994f886-6eb3-4a1a-b60e-489636b44f46 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:48.191273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:48.261770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb77ba0-638e-4786-bd40-76f84229902d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:48.291441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9994f886-6eb3-4a1a-b60e-489636b44f46 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:48.296741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd597a24-efd1-434e-b999-ec910f903cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:48.297617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd597a24-efd1-434e-b999-ec910f903cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:48.298105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd597a24-efd1-434e-b999-ec910f903cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:48.298548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd597a24-efd1-434e-b999-ec910f903cf4 None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:48.299968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd597a24-efd1-434e-b999-ec910f903cf4 None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:48.301768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd597a24-efd1-434e-b999-ec910f903cf4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:48.303607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 226/444] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:48 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:48.310150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd597a24-efd1-434e-b999-ec910f903cf4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:48.310849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:48.332856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9513b-4868-473c-a941-845986bb8341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:48.333497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9513b-4868-473c-a941-845986bb8341 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:48.333795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9513b-4868-473c-a941-845986bb8341 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:48.334064 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f9513b-4868-473c-a941-845986bb8341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:48.405556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9513b-4868-473c-a941-845986bb8341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:48.407049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f9513b-4868-473c-a941-845986bb8341 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:48.418700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd597a24-efd1-434e-b999-ec910f903cf4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:48.429965 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:48.455336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020c613-d8b3-4526-8d5b-b5e7f00f4d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:48.455939 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020c613-d8b3-4526-8d5b-b5e7f00f4d6f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:48.456221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020c613-d8b3-4526-8d5b-b5e7f00f4d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:48.456493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020c613-d8b3-4526-8d5b-b5e7f00f4d6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:48.457245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020c613-d8b3-4526-8d5b-b5e7f00f4d6f None admin] RBAC: Adding query filter params (group.id=ec1800a99f56492687b58a547023a1c3, role.id=dea086603b04451680a14ae806205d79, scope.project.id=8087b41e58fb4aca97cbfced2c011450) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:48.457560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020c613-d8b3-4526-8d5b-b5e7f00f4d6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:48.459326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020c613-d8b3-4526-8d5b-b5e7f00f4d6f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:48.466380 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 227/445] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 22 21:56:48 2022] GET /identity/v3/role_assignments?group.id=ec1800a99f56492687b58a547023a1c3&scope.project.id=8087b41e58fb4aca97cbfced2c011450&role.id=dea086603b04451680a14ae806205d79 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:48.467341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020c613-d8b3-4526-8d5b-b5e7f00f4d6f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:48.525054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 219/446] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:48.526014 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9513b-4868-473c-a941-845986bb8341 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:48.542771 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:48.585781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020c613-d8b3-4526-8d5b-b5e7f00f4d6f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:48.627163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c6b61-8304-48d9-925d-dc45dbc78a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:48.627801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c6b61-8304-48d9-925d-dc45dbc78a8a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:48.628087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c6b61-8304-48d9-925d-dc45dbc78a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:48.628343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c6b61-8304-48d9-925d-dc45dbc78a8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:48.638154 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f9513b-4868-473c-a941-845986bb8341 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:48.639984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c6b61-8304-48d9-925d-dc45dbc78a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b07c6b61-8304-48d9-925d-dc45dbc78a8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:48.645233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:48.646105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:48.646891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 228/447] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:56:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:48.651999 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c6b61-8304-48d9-925d-dc45dbc78a8a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:48.682463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f185a54-9e2f-4b98-a262-41ec9d91edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:48.683093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f185a54-9e2f-4b98-a262-41ec9d91edb3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:48.683373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f185a54-9e2f-4b98-a262-41ec9d91edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:48.683675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f185a54-9e2f-4b98-a262-41ec9d91edb3 None admin] PATH_INFO: `/v3/projects/a36c670998e84f8ba575f12eebf55432` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:48.684824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f185a54-9e2f-4b98-a262-41ec9d91edb3 None admin] RBAC: Authorizing `identity:get_project(project_id=a36c670998e84f8ba575f12eebf55432)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:48.686331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f185a54-9e2f-4b98-a262-41ec9d91edb3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:48.687950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 220/448] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:48 2022] GET /identity/v3/projects/a36c670998e84f8ba575f12eebf55432 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:48.691420 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:48.693417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f185a54-9e2f-4b98-a262-41ec9d91edb3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:48.769371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c6b61-8304-48d9-925d-dc45dbc78a8a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:48.797779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:48.811902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcbcc6-e171-4a69-be04-b3603d75941d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:48.812893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbcc6-e171-4a69-be04-b3603d75941d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:48.813523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbcc6-e171-4a69-be04-b3603d75941d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:48.813799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcbcc6-e171-4a69-be04-b3603d75941d None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:48.814628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcbcc6-e171-4a69-be04-b3603d75941d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:48.814928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcbcc6-e171-4a69-be04-b3603d75941d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:48.816294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcbcc6-e171-4a69-be04-b3603d75941d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:48.819812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f185a54-9e2f-4b98-a262-41ec9d91edb3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:48.826024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 229/449] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:48.831692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcbcc6-e171-4a69-be04-b3603d75941d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:48.860550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c69cb-834d-4009-81e8-05ffe74b6505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:48.861199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c69cb-834d-4009-81e8-05ffe74b6505 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:48.861476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c69cb-834d-4009-81e8-05ffe74b6505 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:48.861778 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3c69cb-834d-4009-81e8-05ffe74b6505 None admin] PATH_INFO: `/v3/groups/ec1800a99f56492687b58a547023a1c3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:48.862850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c69cb-834d-4009-81e8-05ffe74b6505 None admin] RBAC: Authorizing `identity:get_group(group_id=ec1800a99f56492687b58a547023a1c3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:48.864587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3c69cb-834d-4009-81e8-05ffe74b6505 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:48.866231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 221/450] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:48 2022] GET /identity/v3/groups/ec1800a99f56492687b58a547023a1c3 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:48.872634 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c69cb-834d-4009-81e8-05ffe74b6505 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:48.944799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:48.953056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcbcc6-e171-4a69-be04-b3603d75941d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:48.986101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3c69cb-834d-4009-81e8-05ffe74b6505 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:48.992984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24271faf-25af-482e-ae64-a2dd02f91eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:48.993626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24271faf-25af-482e-ae64-a2dd02f91eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:48.993900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24271faf-25af-482e-ae64-a2dd02f91eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:48.994171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24271faf-25af-482e-ae64-a2dd02f91eb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:49.022487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb46ddd-bf4e-4db2-86be-11c3d0d0fd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:49.023101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb46ddd-bf4e-4db2-86be-11c3d0d0fd60 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:49.023381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb46ddd-bf4e-4db2-86be-11c3d0d0fd60 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:49.023655 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb46ddd-bf4e-4db2-86be-11c3d0d0fd60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:49.024376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb46ddd-bf4e-4db2-86be-11c3d0d0fd60 None admin] RBAC: Adding query filter params (group.id=ec1800a99f56492687b58a547023a1c3, role.id=dea086603b04451680a14ae806205d79, scope.project.id=a36c670998e84f8ba575f12eebf55432) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:49.024689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb46ddd-bf4e-4db2-86be-11c3d0d0fd60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:49.026163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb46ddd-bf4e-4db2-86be-11c3d0d0fd60 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:49.032738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 222/451] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 22 21:56:48 2022] GET /identity/v3/role_assignments?group.id=ec1800a99f56492687b58a547023a1c3&scope.project.id=a36c670998e84f8ba575f12eebf55432&role.id=dea086603b04451680a14ae806205d79 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:49.068068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24271faf-25af-482e-ae64-a2dd02f91eb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:49.069601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24271faf-25af-482e-ae64-a2dd02f91eb3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:49.171831 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:49.179521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 230/452] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 21:56:49.185795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb46ddd-bf4e-4db2-86be-11c3d0d0fd60 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:49.202736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:49.287629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb46ddd-bf4e-4db2-86be-11c3d0d0fd60 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:49.320825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d984c0a-0958-4517-b9f1-67623ffa479b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:49.321404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d984c0a-0958-4517-b9f1-67623ffa479b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:49.321694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d984c0a-0958-4517-b9f1-67623ffa479b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:49.321946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d984c0a-0958-4517-b9f1-67623ffa479b None admin] PATH_INFO: `/v3/projects/f680186a14a640efb3f020f909ac5fa9` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:49.323059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d984c0a-0958-4517-b9f1-67623ffa479b None admin] RBAC: Authorizing `identity:get_project(project_id=f680186a14a640efb3f020f909ac5fa9)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:49.325051 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d984c0a-0958-4517-b9f1-67623ffa479b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:49.326677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 223/453] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:56:49 2022] GET /identity/v3/projects/f680186a14a640efb3f020f909ac5fa9 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:49.332242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24271faf-25af-482e-ae64-a2dd02f91eb3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:49.431706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24271faf-25af-482e-ae64-a2dd02f91eb3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:49.464163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395baca6-2a73-4867-bf3f-6078f3f1028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:49.465437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395baca6-2a73-4867-bf3f-6078f3f1028c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:49.465744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395baca6-2a73-4867-bf3f-6078f3f1028c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:49.466061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395baca6-2a73-4867-bf3f-6078f3f1028c None admin] PATH_INFO: `/v3/groups/6704eea971884cad90ddb3b8834405ab` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:49.467414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395baca6-2a73-4867-bf3f-6078f3f1028c None admin] RBAC: Authorizing `identity:get_group(group_id=6704eea971884cad90ddb3b8834405ab)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:49.468835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395baca6-2a73-4867-bf3f-6078f3f1028c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:49.470417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 231/454] 10.222.0.25 () {56 vars in 1180 bytes} [Thu Sep 22 21:56:49 2022] GET /identity/v3/groups/6704eea971884cad90ddb3b8834405ab => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:49.476201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d984c0a-0958-4517-b9f1-67623ffa479b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:49.555045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:49.572393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d984c0a-0958-4517-b9f1-67623ffa479b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:49.607673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ca094-c6a2-471d-b13f-7300b6d6fa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:49.608267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ca094-c6a2-471d-b13f-7300b6d6fa25 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:49.608543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ca094-c6a2-471d-b13f-7300b6d6fa25 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:49.608813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ca094-c6a2-471d-b13f-7300b6d6fa25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:49.609545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ca094-c6a2-471d-b13f-7300b6d6fa25 None admin] RBAC: Adding query filter params (group.id=6704eea971884cad90ddb3b8834405ab, role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=f680186a14a640efb3f020f909ac5fa9) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:49.609847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ca094-c6a2-471d-b13f-7300b6d6fa25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:49.611258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ca094-c6a2-471d-b13f-7300b6d6fa25 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:49.616727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 224/455] 10.222.0.25 () {56 vars in 1376 bytes} [Thu Sep 22 21:56:49 2022] GET /identity/v3/role_assignments?group.id=6704eea971884cad90ddb3b8834405ab&scope.project.id=f680186a14a640efb3f020f909ac5fa9&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:52.395006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22128c-3fb1-42ca-8e40-361c14e8b15c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:52.395489 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22128c-3fb1-42ca-8e40-361c14e8b15c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:52.395489 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc22128c-3fb1-42ca-8e40-361c14e8b15c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:52.397179 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 232/456] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:52.402674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2ce85-78a8-4ad9-b03c-67e8013b4232 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:52.403105 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2ce85-78a8-4ad9-b03c-67e8013b4232 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:52.403284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e2ce85-78a8-4ad9-b03c-67e8013b4232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:52.426099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c9e2ce85-78a8-4ad9-b03c-67e8013b4232 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:52.427289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c9e2ce85-78a8-4ad9-b03c-67e8013b4232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:52.500499 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1ca9f-7e14-4931-a1f8-fcaaff569ef6 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:52.501301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1ca9f-7e14-4931-a1f8-fcaaff569ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:52.501781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e1ca9f-7e14-4931-a1f8-fcaaff569ef6 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:52.503795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 233/457] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:52.510585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fde901-ded6-46a8-95f1-b1ce5adcb753 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:52.511088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fde901-ded6-46a8-95f1-b1ce5adcb753 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:52.511725 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fde901-ded6-46a8-95f1-b1ce5adcb753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:52.523172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 225/458] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 22 21:56:52.546144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-46fde901-ded6-46a8-95f1-b1ce5adcb753 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:52.547347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-46fde901-ded6-46a8-95f1-b1ce5adcb753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:52.620196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:52.643564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 234/459] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:52.738268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00d5ff-058a-47ae-87e2-964b78c2a7ed None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:52.738559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00d5ff-058a-47ae-87e2-964b78c2a7ed None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:52.738724 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb00d5ff-058a-47ae-87e2-964b78c2a7ed None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:52.740353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 226/460] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:52.745173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f5d66-1827-4195-83b5-19398544315e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:52.745460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f5d66-1827-4195-83b5-19398544315e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:52.745870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f5d66-1827-4195-83b5-19398544315e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:52.774135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-258f5d66-1827-4195-83b5-19398544315e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:52.775318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-258f5d66-1827-4195-83b5-19398544315e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:52.862591 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 235/461] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:52.883784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00d5ff-058a-47ae-87e2-964b78c2a7ed None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:52.991589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:53.028725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb00d5ff-058a-47ae-87e2-964b78c2a7ed None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:53.076795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d44f0e-b42f-41d6-ba4c-16fc29236f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:53.077453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d44f0e-b42f-41d6-ba4c-16fc29236f2f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.077783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d44f0e-b42f-41d6-ba4c-16fc29236f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.078059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d44f0e-b42f-41d6-ba4c-16fc29236f2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.080254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f5d66-1827-4195-83b5-19398544315e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:53.098454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d44f0e-b42f-41d6-ba4c-16fc29236f2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:53.100135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d44f0e-b42f-41d6-ba4c-16fc29236f2f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-12d44f0e-b42f-41d6-ba4c-16fc29236f2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:56:53.130783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:53.132016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:56:53.133124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:56:53.133124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:53.133124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:53.134669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 227/462] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:53.136088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c6d79-79ce-4cad-97a1-2ef8f11da77f None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.136425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c6d79-79ce-4cad-97a1-2ef8f11da77f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.137208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0c6d79-79ce-4cad-97a1-2ef8f11da77f None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.138903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 228/463] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:53.144057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c6d79-79ce-4cad-97a1-2ef8f11da77f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:53.204951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f5d66-1827-4195-83b5-19398544315e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:53.231438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:53.242885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee94d0-4a21-4c3e-ac1f-2b80b59a8df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:53.243795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee94d0-4a21-4c3e-ac1f-2b80b59a8df1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.244264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee94d0-4a21-4c3e-ac1f-2b80b59a8df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.244707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee94d0-4a21-4c3e-ac1f-2b80b59a8df1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.255110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee94d0-4a21-4c3e-ac1f-2b80b59a8df1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:53.256768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee94d0-4a21-4c3e-ac1f-2b80b59a8df1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:53.261863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0c6d79-79ce-4cad-97a1-2ef8f11da77f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-24ee94d0-4a21-4c3e-ac1f-2b80b59a8df1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:56:53.272785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:53.273720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:56:53.274544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:56:53.274544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:53.274544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:53.274544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 236/464] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:53.275445 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ce7fc-010b-4618-8736-d954088064bb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.275445 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ce7fc-010b-4618-8736-d954088064bb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.275608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ce7fc-010b-4618-8736-d954088064bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.294031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b30ce7fc-010b-4618-8736-d954088064bb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:53.295260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b30ce7fc-010b-4618-8736-d954088064bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:53.296944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:53.307417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64b767-c60f-4866-ad17-aa524de3e7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:53.307919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64b767-c60f-4866-ad17-aa524de3e7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.308663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64b767-c60f-4866-ad17-aa524de3e7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.308941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed64b767-c60f-4866-ad17-aa524de3e7d2 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.309614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64b767-c60f-4866-ad17-aa524de3e7d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:53.309916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64b767-c60f-4866-ad17-aa524de3e7d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:53.311322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed64b767-c60f-4866-ad17-aa524de3e7d2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:53.323648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 229/465] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:53.324914 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64b767-c60f-4866-ad17-aa524de3e7d2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:53.371691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 237/466] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:53.372820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ce7fc-010b-4618-8736-d954088064bb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:53.457252 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed64b767-c60f-4866-ad17-aa524de3e7d2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:53.485436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ce7fc-010b-4618-8736-d954088064bb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:53.494952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:53.503229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c884325-ede7-41c5-a2ef-121c1dead0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:53.503956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c884325-ede7-41c5-a2ef-121c1dead0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.504531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c884325-ede7-41c5-a2ef-121c1dead0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.504875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c884325-ede7-41c5-a2ef-121c1dead0f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.519144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c884325-ede7-41c5-a2ef-121c1dead0f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:53.521344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c884325-ede7-41c5-a2ef-121c1dead0f7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:53.530801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75492b2-d4b0-4f42-ad99-89ea13123447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:53.531637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75492b2-d4b0-4f42-ad99-89ea13123447 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.532449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75492b2-d4b0-4f42-ad99-89ea13123447 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.533226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75492b2-d4b0-4f42-ad99-89ea13123447 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.534042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75492b2-d4b0-4f42-ad99-89ea13123447 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:53.534515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75492b2-d4b0-4f42-ad99-89ea13123447 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:53.536029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75492b2-d4b0-4f42-ad99-89ea13123447 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0c884325-ede7-41c5-a2ef-121c1dead0f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:56:53.539385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:53.540253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:56:53.541271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:56:53.541271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:53.541271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:53.541271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 230/467] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:53.542138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c884325-ede7-41c5-a2ef-121c1dead0f7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:53.552067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 238/468] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:53.553254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75492b2-d4b0-4f42-ad99-89ea13123447 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:53.600567 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:53.678313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75492b2-d4b0-4f42-ad99-89ea13123447 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:53.681610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c884325-ede7-41c5-a2ef-121c1dead0f7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:53.717989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063793f-5bdd-42a5-b53a-a7e35c2362e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:53.718592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063793f-5bdd-42a5-b53a-a7e35c2362e1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.718884 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063793f-5bdd-42a5-b53a-a7e35c2362e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.719130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9063793f-5bdd-42a5-b53a-a7e35c2362e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.719763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063793f-5bdd-42a5-b53a-a7e35c2362e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:53.720059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063793f-5bdd-42a5-b53a-a7e35c2362e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:53.720895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25753-9706-401f-8413-88fd6019da95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:53.721431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9063793f-5bdd-42a5-b53a-a7e35c2362e1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:53.721528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25753-9706-401f-8413-88fd6019da95 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.721794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25753-9706-401f-8413-88fd6019da95 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.722075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd25753-9706-401f-8413-88fd6019da95 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.722695 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25753-9706-401f-8413-88fd6019da95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:53.724727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd25753-9706-401f-8413-88fd6019da95 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:53.733984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 239/469] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:56:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:53.734585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063793f-5bdd-42a5-b53a-a7e35c2362e1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:53.750893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 231/470] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:56:53 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:56:53.751753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25753-9706-401f-8413-88fd6019da95 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:53.769594 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:53.858291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9063793f-5bdd-42a5-b53a-a7e35c2362e1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:53.862926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:53.871480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd25753-9706-401f-8413-88fd6019da95 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:53.898182 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d774b92-fadb-44b7-9242-2061a237f5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:53.898799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d774b92-fadb-44b7-9242-2061a237f5e0 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.899071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d774b92-fadb-44b7-9242-2061a237f5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.899335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d774b92-fadb-44b7-9242-2061a237f5e0 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.899985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d774b92-fadb-44b7-9242-2061a237f5e0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:53.901469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d774b92-fadb-44b7-9242-2061a237f5e0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:53.912949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b74dc-9789-4df9-b624-ba92786fc521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:53.913853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b74dc-9789-4df9-b624-ba92786fc521 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.913971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b74dc-9789-4df9-b624-ba92786fc521 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.914280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b74dc-9789-4df9-b624-ba92786fc521 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.925883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 240/471] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:56:53 2022] POST /identity/v3/users => generated 246 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:56:53.926240 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b74dc-9789-4df9-b624-ba92786fc521 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:53.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d774b92-fadb-44b7-9242-2061a237f5e0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:53.927772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b74dc-9789-4df9-b624-ba92786fc521 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-698b74dc-9789-4df9-b624-ba92786fc521 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:56:53.944235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:53.945228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:56:53.947573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:56:53.947573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:53.947573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:53.947573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 232/472] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:53.947573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80417f08-af16-4101-8c81-9bb84eb80d42 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.947573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80417f08-af16-4101-8c81-9bb84eb80d42 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.947573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80417f08-af16-4101-8c81-9bb84eb80d42 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.950758 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 233/473] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:53.957278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527e438-0a50-4a2e-b7a7-5617bfc1555c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:53.957577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527e438-0a50-4a2e-b7a7-5617bfc1555c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:53.957832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527e438-0a50-4a2e-b7a7-5617bfc1555c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:53.985533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a527e438-0a50-4a2e-b7a7-5617bfc1555c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:53.986789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a527e438-0a50-4a2e-b7a7-5617bfc1555c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:54.055433 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:54.081057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d774b92-fadb-44b7-9242-2061a237f5e0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:54.098736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 234/474] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:54.099853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527e438-0a50-4a2e-b7a7-5617bfc1555c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:54.151318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729b150-6fb2-439f-8b00-3408fe303a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:54.152319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b150-6fb2-439f-8b00-3408fe303a59 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:54.152830 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b150-6fb2-439f-8b00-3408fe303a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:54.153341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729b150-6fb2-439f-8b00-3408fe303a59 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:54.154216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729b150-6fb2-439f-8b00-3408fe303a59 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:54.155872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729b150-6fb2-439f-8b00-3408fe303a59 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:54.184372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 241/475] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:56:53 2022] POST /identity/v3/users => generated 248 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:56:54.215559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:54.231562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527e438-0a50-4a2e-b7a7-5617bfc1555c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:54.274499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320444f5-635c-42b7-9d9d-71d224605ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:54.275206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320444f5-635c-42b7-9d9d-71d224605ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:54.275481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320444f5-635c-42b7-9d9d-71d224605ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:54.275752 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320444f5-635c-42b7-9d9d-71d224605ad0 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:54.276471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320444f5-635c-42b7-9d9d-71d224605ad0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:54.276779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320444f5-635c-42b7-9d9d-71d224605ad0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:54.278391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320444f5-635c-42b7-9d9d-71d224605ad0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:54.295950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 235/476] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:56:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:54.302252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729b150-6fb2-439f-8b00-3408fe303a59 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:54.445420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320444f5-635c-42b7-9d9d-71d224605ad0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:54.446884 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729b150-6fb2-439f-8b00-3408fe303a59 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:54.488772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c2ce8-5433-4f60-a98c-6e8413c23a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:54.489426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2ce8-5433-4f60-a98c-6e8413c23a66 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:54.489708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2ce8-5433-4f60-a98c-6e8413c23a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:54.489979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c2ce8-5433-4f60-a98c-6e8413c23a66 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:54.490619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c2ce8-5433-4f60-a98c-6e8413c23a66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:54.492137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c2ce8-5433-4f60-a98c-6e8413c23a66 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:54.505430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:54.518011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 242/477] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:56:54 2022] POST /identity/v3/users => generated 249 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:56:54.574734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320444f5-635c-42b7-9d9d-71d224605ad0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:54.615611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b34942-a1fc-4078-abc6-87c3ab238dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:54.616226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b34942-a1fc-4078-abc6-87c3ab238dd9 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:54.616504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b34942-a1fc-4078-abc6-87c3ab238dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:54.616770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b34942-a1fc-4078-abc6-87c3ab238dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:54.617499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b34942-a1fc-4078-abc6-87c3ab238dd9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:54.618902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b34942-a1fc-4078-abc6-87c3ab238dd9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:54.628004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.sql.core [None req-77b34942-a1fc-4078-abc6-87c3ab238dd9 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 21:56:54.628004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 21:56:54.628004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [parameters: {'id': '806e653f2d2249a283b6ed23c93dd302', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 21:56:54.628004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81628) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-77b34942-a1fc-4078-abc6-87c3ab238dd9 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application conn.query(q) Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result.read() Sep 22 21:56:54.645538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return ref.to_dict() Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 21:56:54.646561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application next(self.gen) Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application yield resource Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application next(self.gen) Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise self.value Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application session.commit() Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 21:56:54.647736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.session.flush() Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application compat.raise_( Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 21:56:54.648869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 21:56:54.649767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application conn.query(q) Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result.read() Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 21:56:54.650551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [parameters: {'id': '806e653f2d2249a283b6ed23c93dd302', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:54.651339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 22 21:56:54.654023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:54.655043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 236/478] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 22 21:56:54 2022] POST /identity/v3/roles => generated 144 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:56:54.655121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c2ce8-5433-4f60-a98c-6e8413c23a66 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:54.784154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c2ce8-5433-4f60-a98c-6e8413c23a66 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:54.819660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e08cb-71e3-41a7-acb6-52f0009ab97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:54.820294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e08cb-71e3-41a7-acb6-52f0009ab97b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:54.820568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e08cb-71e3-41a7-acb6-52f0009ab97b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:54.820883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e08cb-71e3-41a7-acb6-52f0009ab97b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:54.831698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e08cb-71e3-41a7-acb6-52f0009ab97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-949e08cb-71e3-41a7-acb6-52f0009ab97b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:54.836919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:54.838070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:54.839158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 243/479] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:56:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:54.846187 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b34942-a1fc-4078-abc6-87c3ab238dd9 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:54.849249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:54.960264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b34942-a1fc-4078-abc6-87c3ab238dd9 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:54.999085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80eefbe-4876-43ae-8de0-44600043afb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:54.999751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eefbe-4876-43ae-8de0-44600043afb2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:55.000028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eefbe-4876-43ae-8de0-44600043afb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:55.000297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80eefbe-4876-43ae-8de0-44600043afb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:55.000952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80eefbe-4876-43ae-8de0-44600043afb2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:55.001282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80eefbe-4876-43ae-8de0-44600043afb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:55.002658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80eefbe-4876-43ae-8de0-44600043afb2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:55.012349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 237/480] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:56:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:56.474595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cf44f-5a91-400a-92a9-cdffde00b295 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:56.475091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cf44f-5a91-400a-92a9-cdffde00b295 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:56.475091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3cf44f-5a91-400a-92a9-cdffde00b295 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:56.476620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 244/481] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:56.482532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d04aa8-c0a9-4ffc-81b1-21c2b579d553 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:56.483145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d04aa8-c0a9-4ffc-81b1-21c2b579d553 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:56.483145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d04aa8-c0a9-4ffc-81b1-21c2b579d553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:56.515494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-81d04aa8-c0a9-4ffc-81b1-21c2b579d553 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:56.516679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-81d04aa8-c0a9-4ffc-81b1-21c2b579d553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:56.600377 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:56.612941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 238/482] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:56.618781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e4ef6-dba3-45d9-bf6e-9547fd1f43c3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:56.619063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e4ef6-dba3-45d9-bf6e-9547fd1f43c3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:56.619341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e4ef6-dba3-45d9-bf6e-9547fd1f43c3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:56.620689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 245/483] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:56.625886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e98d23-2473-47ce-84ac-e4a963a4e045 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:56.626170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e98d23-2473-47ce-84ac-e4a963a4e045 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:56.626438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e98d23-2473-47ce-84ac-e4a963a4e045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:56.654125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-63e98d23-2473-47ce-84ac-e4a963a4e045 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:56.655325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-63e98d23-2473-47ce-84ac-e4a963a4e045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:56.749078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 239/484] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:56.973406 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e4ef6-dba3-45d9-bf6e-9547fd1f43c3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:57.038173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:57.090985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e4ef6-dba3-45d9-bf6e-9547fd1f43c3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:57.126357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea9ea7-2d54-4099-a229-d68018af9206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:57.126984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9ea7-2d54-4099-a229-d68018af9206 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:57.127259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9ea7-2d54-4099-a229-d68018af9206 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:57.127530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea9ea7-2d54-4099-a229-d68018af9206 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:57.139290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea9ea7-2d54-4099-a229-d68018af9206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-16ea9ea7-2d54-4099-a229-d68018af9206 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:57.145177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:57.146102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:56:57.146972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:57.148182 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 246/485] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:56:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:57.154207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e98d23-2473-47ce-84ac-e4a963a4e045 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:57.179453 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0e95-073f-4d7a-a665-4fd1cbd86097 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:57.180070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0e95-073f-4d7a-a665-4fd1cbd86097 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:57.180537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c0e95-073f-4d7a-a665-4fd1cbd86097 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:57.182412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 247/486] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:56:57.187712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6ed0c-d82a-4406-bf0d-1ee23f2b5cb3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:57.187995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6ed0c-d82a-4406-bf0d-1ee23f2b5cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:57.188267 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6ed0c-d82a-4406-bf0d-1ee23f2b5cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:57.207859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-dbc6ed0c-d82a-4406-bf0d-1ee23f2b5cb3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:57.209426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dbc6ed0c-d82a-4406-bf0d-1ee23f2b5cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:57.289663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 248/487] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:56:57.290791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6ed0c-d82a-4406-bf0d-1ee23f2b5cb3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:57.330720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e98d23-2473-47ce-84ac-e4a963a4e045 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:57.359590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:57.385907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e08b4-ab6b-4adf-a7df-1cb16acc9250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:57.387289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e08b4-ab6b-4adf-a7df-1cb16acc9250 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:57.387735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e08b4-ab6b-4adf-a7df-1cb16acc9250 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:57.388145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e08b4-ab6b-4adf-a7df-1cb16acc9250 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:57.388942 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e08b4-ab6b-4adf-a7df-1cb16acc9250 None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:57.390506 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e08b4-ab6b-4adf-a7df-1cb16acc9250 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:57.393258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57e08b4-ab6b-4adf-a7df-1cb16acc9250 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:57.397475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:57.412708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 240/488] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:56:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:57.418158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e08b4-ab6b-4adf-a7df-1cb16acc9250 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:57.475517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6ed0c-d82a-4406-bf0d-1ee23f2b5cb3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:57.541513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fa1b7-6232-4e86-a40d-0fc55125046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:57.543019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fa1b7-6232-4e86-a40d-0fc55125046c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:57.543464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fa1b7-6232-4e86-a40d-0fc55125046c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:57.543878 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fa1b7-6232-4e86-a40d-0fc55125046c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:57.566427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0fa1b7-6232-4e86-a40d-0fc55125046c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bd0fa1b7-6232-4e86-a40d-0fc55125046c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:57.572553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:57.573664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:56:57.574474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:57.578543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 249/489] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:56:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:57.583996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fa1b7-6232-4e86-a40d-0fc55125046c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:57.612973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e08b4-ab6b-4adf-a7df-1cb16acc9250 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:57.655302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d9cf1-8bc6-414a-b355-9e5158ca27cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:57.656276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d9cf1-8bc6-414a-b355-9e5158ca27cd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:57.656565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d9cf1-8bc6-414a-b355-9e5158ca27cd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:57.656893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d9cf1-8bc6-414a-b355-9e5158ca27cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:57.670967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d9cf1-8bc6-414a-b355-9e5158ca27cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:57.672489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81d9cf1-8bc6-414a-b355-9e5158ca27cd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e81d9cf1-8bc6-414a-b355-9e5158ca27cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:56:57.686372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:57.687283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:56:57.693485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:56:57.693485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:57.693485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:57.693485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 241/490] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:57.693485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d9cf1-8bc6-414a-b355-9e5158ca27cd None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:57.722924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fa1b7-6232-4e86-a40d-0fc55125046c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:57.727001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:57.736892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:57.765021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d0583-37f1-4b32-9302-85f82d09671b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:57.765683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d0583-37f1-4b32-9302-85f82d09671b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:57.765959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d0583-37f1-4b32-9302-85f82d09671b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:57.766228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d0583-37f1-4b32-9302-85f82d09671b None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:57.766883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d0583-37f1-4b32-9302-85f82d09671b None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:57.767190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d0583-37f1-4b32-9302-85f82d09671b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:57.768548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d0583-37f1-4b32-9302-85f82d09671b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:57.777657 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 250/491] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:56:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:56:57.782977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439fe1c-7cb5-4b2c-8806-bcf091413f03 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:57.783266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439fe1c-7cb5-4b2c-8806-bcf091413f03 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:57.783544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439fe1c-7cb5-4b2c-8806-bcf091413f03 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:57.785090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 251/492] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:57.789731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439fe1c-7cb5-4b2c-8806-bcf091413f03 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:57.808760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d9cf1-8bc6-414a-b355-9e5158ca27cd None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:57.842718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703eca7-8492-41e5-b028-6035ab5a4e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:57.843766 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703eca7-8492-41e5-b028-6035ab5a4e9d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:57.844113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703eca7-8492-41e5-b028-6035ab5a4e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:57.844454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5703eca7-8492-41e5-b028-6035ab5a4e9d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:57.855340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5703eca7-8492-41e5-b028-6035ab5a4e9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-5703eca7-8492-41e5-b028-6035ab5a4e9d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:57.860061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:57.860957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:57.861803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 242/493] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:56:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:57.862455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703eca7-8492-41e5-b028-6035ab5a4e9d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:57.901983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439fe1c-7cb5-4b2c-8806-bcf091413f03 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:57.934674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88778560-f14e-478a-bb8e-68107be23a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:57.935264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88778560-f14e-478a-bb8e-68107be23a8f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:57.935537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88778560-f14e-478a-bb8e-68107be23a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:57.935805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88778560-f14e-478a-bb8e-68107be23a8f None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:57.936429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88778560-f14e-478a-bb8e-68107be23a8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:57.936726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88778560-f14e-478a-bb8e-68107be23a8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:57.938121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88778560-f14e-478a-bb8e-68107be23a8f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:57.949232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 252/494] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:57.950360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25175f6-2d5b-46c3-bdf3-f90c7d22bd53 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:57.950634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25175f6-2d5b-46c3-bdf3-f90c7d22bd53 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:57.950899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25175f6-2d5b-46c3-bdf3-f90c7d22bd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:57.965441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:57.968213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a25175f6-2d5b-46c3-bdf3-f90c7d22bd53 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:57.969462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a25175f6-2d5b-46c3-bdf3-f90c7d22bd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:57.969571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5703eca7-8492-41e5-b028-6035ab5a4e9d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:57.990169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:58.008876 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af996413-53dc-4834-bb04-8f9179ebee5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:58.009614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af996413-53dc-4834-bb04-8f9179ebee5a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.009895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af996413-53dc-4834-bb04-8f9179ebee5a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.010158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af996413-53dc-4834-bb04-8f9179ebee5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.020469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af996413-53dc-4834-bb04-8f9179ebee5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:58.021938 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af996413-53dc-4834-bb04-8f9179ebee5a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-af996413-53dc-4834-bb04-8f9179ebee5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:56:58.036667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:58.037790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:56:58.038643 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:56:58.038643 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:58.038643 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:58.038643 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 243/495] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:58.038901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5fa44-2f18-4d22-a4f6-363dafa1b09d None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.039177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5fa44-2f18-4d22-a4f6-363dafa1b09d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.039441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d5fa44-2f18-4d22-a4f6-363dafa1b09d None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.045027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 244/496] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:56:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:56:58.045027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5fa44-2f18-4d22-a4f6-363dafa1b09d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:58.045250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 253/497] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:58.045250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25175f6-2d5b-46c3-bdf3-f90c7d22bd53 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:58.150884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d5fa44-2f18-4d22-a4f6-363dafa1b09d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:58.153747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25175f6-2d5b-46c3-bdf3-f90c7d22bd53 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:58.186002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a93a8d-d01f-41a8-b458-5e089614b38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:58.186805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93a8d-d01f-41a8-b458-5e089614b38e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.187224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93a8d-d01f-41a8-b458-5e089614b38e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.187633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93a8d-d01f-41a8-b458-5e089614b38e None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.188453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a93a8d-d01f-41a8-b458-5e089614b38e None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:58.188890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a93a8d-d01f-41a8-b458-5e089614b38e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:58.189324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c211e565-7b78-4b16-91cd-2c1b8afc55c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:58.189939 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211e565-7b78-4b16-91cd-2c1b8afc55c4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.190212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211e565-7b78-4b16-91cd-2c1b8afc55c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.190476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c211e565-7b78-4b16-91cd-2c1b8afc55c4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.191080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a93a8d-d01f-41a8-b458-5e089614b38e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:58.197969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211e565-7b78-4b16-91cd-2c1b8afc55c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:58.199593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c211e565-7b78-4b16-91cd-2c1b8afc55c4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:58.199901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 245/498] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:56:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:58.200854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a93a8d-d01f-41a8-b458-5e089614b38e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c211e565-7b78-4b16-91cd-2c1b8afc55c4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:56:58.219224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:58.220362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:58.221516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:58.221516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:58.221516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:56:58.221516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:56:58.221516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:56:58.221516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:56:58.221516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:56:58.221516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:56:58.221516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 21:56:58.221516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:58.223629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:58.226394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 254/499] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:58.227369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40990b8d-131c-4043-ac45-1a315f5d389e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.227653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40990b8d-131c-4043-ac45-1a315f5d389e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.227928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40990b8d-131c-4043-ac45-1a315f5d389e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.246747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-40990b8d-131c-4043-ac45-1a315f5d389e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:56:58.247930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-40990b8d-131c-4043-ac45-1a315f5d389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:56:58.259646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:58.313358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a93a8d-d01f-41a8-b458-5e089614b38e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:58.318153 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 255/500] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:56:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:56:58.318996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40990b8d-131c-4043-ac45-1a315f5d389e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:58.359215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0410-c30d-4e4f-8a36-cd8e4f7b17f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:58.359819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0410-c30d-4e4f-8a36-cd8e4f7b17f0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.360097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0410-c30d-4e4f-8a36-cd8e4f7b17f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.360366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffe0410-c30d-4e4f-8a36-cd8e4f7b17f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.360996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0410-c30d-4e4f-8a36-cd8e4f7b17f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:58.361325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0410-c30d-4e4f-8a36-cd8e4f7b17f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:58.362675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffe0410-c30d-4e4f-8a36-cd8e4f7b17f0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:58.379440 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 246/501] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:58.379440 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0410-c30d-4e4f-8a36-cd8e4f7b17f0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:58.427887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40990b8d-131c-4043-ac45-1a315f5d389e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:58.444857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:58.467970 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c452b5-73fd-4ff7-8a3e-881ab0bb361e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:58.468976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c452b5-73fd-4ff7-8a3e-881ab0bb361e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.469278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c452b5-73fd-4ff7-8a3e-881ab0bb361e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.469548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c452b5-73fd-4ff7-8a3e-881ab0bb361e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.474395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffe0410-c30d-4e4f-8a36-cd8e4f7b17f0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:58.481312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c452b5-73fd-4ff7-8a3e-881ab0bb361e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:58.482867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c452b5-73fd-4ff7-8a3e-881ab0bb361e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:58.489698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b0c452b5-73fd-4ff7-8a3e-881ab0bb361e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:56:58.500847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:58.501797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:56:58.502630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:56:58.502630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:58.502630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:58.502630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 256/502] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:58.502845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c452b5-73fd-4ff7-8a3e-881ab0bb361e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:58.513294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962168f5-8dec-4ef5-8583-b07549cecbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:58.513895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962168f5-8dec-4ef5-8583-b07549cecbae None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.514170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962168f5-8dec-4ef5-8583-b07549cecbae None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.514439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962168f5-8dec-4ef5-8583-b07549cecbae None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.515155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962168f5-8dec-4ef5-8583-b07549cecbae None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:58.515455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962168f5-8dec-4ef5-8583-b07549cecbae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:58.516953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962168f5-8dec-4ef5-8583-b07549cecbae None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:58.558791 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 247/503] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 22 21:56:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:58.558791 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962168f5-8dec-4ef5-8583-b07549cecbae None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:58.616938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c452b5-73fd-4ff7-8a3e-881ab0bb361e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:58.649608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8f824-6352-49ae-8f8b-350cefa19200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:58.650225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8f824-6352-49ae-8f8b-350cefa19200 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.650506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8f824-6352-49ae-8f8b-350cefa19200 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.650779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8f824-6352-49ae-8f8b-350cefa19200 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.656415 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962168f5-8dec-4ef5-8583-b07549cecbae None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:58.661357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8f824-6352-49ae-8f8b-350cefa19200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-15c8f824-6352-49ae-8f8b-350cefa19200 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:56:58.666515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:58.667654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 257/504] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:56:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:58.668729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8f824-6352-49ae-8f8b-350cefa19200 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:58.687670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:58.693954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9ecf7-0475-49a9-b9b6-009f4f856ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:58.694580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9ecf7-0475-49a9-b9b6-009f4f856ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.694947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9ecf7-0475-49a9-b9b6-009f4f856ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.695274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9ecf7-0475-49a9-b9b6-009f4f856ed3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.703253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9ecf7-0475-49a9-b9b6-009f4f856ed3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:58.704701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9ecf7-0475-49a9-b9b6-009f4f856ed3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-aeb9ecf7-0475-49a9-b9b6-009f4f856ed3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:56:58.718339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:58.719410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 248/505] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 22 21:56:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:58.720294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9ecf7-0475-49a9-b9b6-009f4f856ed3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:58.730369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:58.773394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8f824-6352-49ae-8f8b-350cefa19200 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:58.807017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabefd2b-6f06-4d91-9278-a8063b108754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:58.807625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabefd2b-6f06-4d91-9278-a8063b108754 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.808462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabefd2b-6f06-4d91-9278-a8063b108754 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.808729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabefd2b-6f06-4d91-9278-a8063b108754 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.809400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabefd2b-6f06-4d91-9278-a8063b108754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:58.809705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabefd2b-6f06-4d91-9278-a8063b108754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:58.811081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabefd2b-6f06-4d91-9278-a8063b108754 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:58.818718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9ecf7-0475-49a9-b9b6-009f4f856ed3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:58.822452 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 258/506] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:58.823246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabefd2b-6f06-4d91-9278-a8063b108754 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:58.853590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18e6aa-489c-4fd0-ae09-204573dc75bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:58.854183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18e6aa-489c-4fd0-ae09-204573dc75bb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.854457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18e6aa-489c-4fd0-ae09-204573dc75bb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.855338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18e6aa-489c-4fd0-ae09-204573dc75bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.910847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:58.920927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18e6aa-489c-4fd0-ae09-204573dc75bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:58.922398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e18e6aa-489c-4fd0-ae09-204573dc75bb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:58.923884 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabefd2b-6f06-4d91-9278-a8063b108754 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:58.941156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:58.957634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fa054-76f9-4c3d-8c1a-882f486d628b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:58.958248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fa054-76f9-4c3d-8c1a-882f486d628b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:58.958521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fa054-76f9-4c3d-8c1a-882f486d628b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:58.958790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85fa054-76f9-4c3d-8c1a-882f486d628b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:58.969498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fa054-76f9-4c3d-8c1a-882f486d628b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:58.970918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85fa054-76f9-4c3d-8c1a-882f486d628b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f85fa054-76f9-4c3d-8c1a-882f486d628b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:56:58.985891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:58.986770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:56:58.987645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:56:58.987645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:58.987645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:58.987645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 259/507] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:58.988093 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fa054-76f9-4c3d-8c1a-882f486d628b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.019870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 249/508] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:59.020708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18e6aa-489c-4fd0-ae09-204573dc75bb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.090275 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85fa054-76f9-4c3d-8c1a-882f486d628b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:59.112524 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:59.123186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18e6aa-489c-4fd0-ae09-204573dc75bb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:59.123553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2fa35-4abc-42af-b932-1de3558885cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:59.124155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2fa35-4abc-42af-b932-1de3558885cc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:59.124434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2fa35-4abc-42af-b932-1de3558885cc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:59.124705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f2fa35-4abc-42af-b932-1de3558885cc None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:59.125390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2fa35-4abc-42af-b932-1de3558885cc None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:59.125689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2fa35-4abc-42af-b932-1de3558885cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:59.127057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f2fa35-4abc-42af-b932-1de3558885cc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:59.134802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 260/509] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:56:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:59.135576 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2fa35-4abc-42af-b932-1de3558885cc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.156508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea0045-e7be-4ee9-bd75-bc5e2f98a173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:59.157116 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea0045-e7be-4ee9-bd75-bc5e2f98a173 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:59.157399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea0045-e7be-4ee9-bd75-bc5e2f98a173 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:59.157668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ea0045-e7be-4ee9-bd75-bc5e2f98a173 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:59.158384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea0045-e7be-4ee9-bd75-bc5e2f98a173 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:59.158686 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea0045-e7be-4ee9-bd75-bc5e2f98a173 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:59.160124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ea0045-e7be-4ee9-bd75-bc5e2f98a173 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:59.168169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:59.178649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 250/510] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 22 21:56:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:59.179489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea0045-e7be-4ee9-bd75-bc5e2f98a173 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.239236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f2fa35-4abc-42af-b932-1de3558885cc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:59.272528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30beb6b3-55e9-483e-a45c-78a2075fad51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:59.273170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beb6b3-55e9-483e-a45c-78a2075fad51 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:59.273451 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beb6b3-55e9-483e-a45c-78a2075fad51 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:59.273726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30beb6b3-55e9-483e-a45c-78a2075fad51 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:59.276313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ea0045-e7be-4ee9-bd75-bc5e2f98a173 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:59.280883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30beb6b3-55e9-483e-a45c-78a2075fad51 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:59.282358 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30beb6b3-55e9-483e-a45c-78a2075fad51 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-30beb6b3-55e9-483e-a45c-78a2075fad51 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:56:59.293612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:59.294686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:59.295569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:59.295569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:59.295569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:56:59.295569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:56:59.295569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:56:59.295569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:56:59.295569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:56:59.295569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:56:59.295569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 21:56:59.295569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:59.295569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 261/511] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:56:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:59.296110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30beb6b3-55e9-483e-a45c-78a2075fad51 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.311377 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460c893-e001-4aa1-8b40-96c5f40e0772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:59.311975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460c893-e001-4aa1-8b40-96c5f40e0772 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:59.312248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460c893-e001-4aa1-8b40-96c5f40e0772 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:59.312513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5460c893-e001-4aa1-8b40-96c5f40e0772 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:59.313160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460c893-e001-4aa1-8b40-96c5f40e0772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:59.313460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460c893-e001-4aa1-8b40-96c5f40e0772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:59.314826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5460c893-e001-4aa1-8b40-96c5f40e0772 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:59.325964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 251/512] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:59.326588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460c893-e001-4aa1-8b40-96c5f40e0772 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.338229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:59.389592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:59.400037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30beb6b3-55e9-483e-a45c-78a2075fad51 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:59.425045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5460c893-e001-4aa1-8b40-96c5f40e0772 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:59.433543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56be196-6c1c-4aed-99aa-f195dc28315b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:59.434236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56be196-6c1c-4aed-99aa-f195dc28315b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:59.434513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56be196-6c1c-4aed-99aa-f195dc28315b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:59.434780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56be196-6c1c-4aed-99aa-f195dc28315b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:59.444918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56be196-6c1c-4aed-99aa-f195dc28315b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:59.446383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56be196-6c1c-4aed-99aa-f195dc28315b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:59.457457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2d6c4-eed2-4fdf-ab9e-dd39ae341a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:59.458073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2d6c4-eed2-4fdf-ab9e-dd39ae341a54 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:59.458349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2d6c4-eed2-4fdf-ab9e-dd39ae341a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:59.458618 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c2d6c4-eed2-4fdf-ab9e-dd39ae341a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c56be196-6c1c-4aed-99aa-f195dc28315b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:56:59.459979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:59.460860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:56:59.466693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:56:59.466693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:59.466693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:56:59.466693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 262/513] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:59.466693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56be196-6c1c-4aed-99aa-f195dc28315b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.469679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2d6c4-eed2-4fdf-ab9e-dd39ae341a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:59.471113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c2d6c4-eed2-4fdf-ab9e-dd39ae341a54 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-b4c2d6c4-eed2-4fdf-ab9e-dd39ae341a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:56:59.485528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:56:59.486393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:56:59.487206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:56:59.487206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:56:59.487206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:56:59.487206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 252/514] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:56:59.487435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2d6c4-eed2-4fdf-ab9e-dd39ae341a54 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.545412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:59.566644 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56be196-6c1c-4aed-99aa-f195dc28315b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:59.587701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c2d6c4-eed2-4fdf-ab9e-dd39ae341a54 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:59.599113 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ce1ce-ef11-4ef5-bd38-a0b5b959f680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:59.599730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ce1ce-ef11-4ef5-bd38-a0b5b959f680 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:59.600005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ce1ce-ef11-4ef5-bd38-a0b5b959f680 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:59.600271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ce1ce-ef11-4ef5-bd38-a0b5b959f680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:59.614923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:59.621480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad6c48-703c-460e-b533-ce410ecc7ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:59.622103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad6c48-703c-460e-b533-ce410ecc7ccf None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:59.622378 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad6c48-703c-460e-b533-ce410ecc7ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:59.622637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad6c48-703c-460e-b533-ce410ecc7ccf None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:59.623345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad6c48-703c-460e-b533-ce410ecc7ccf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:59.623646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad6c48-703c-460e-b533-ce410ecc7ccf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:59.625067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daad6c48-703c-460e-b533-ce410ecc7ccf None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:59.641428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 253/515] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 22 21:56:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:59.642197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad6c48-703c-460e-b533-ce410ecc7ccf None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.673933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ce1ce-ef11-4ef5-bd38-a0b5b959f680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:59.675389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ce1ce-ef11-4ef5-bd38-a0b5b959f680 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:59.739508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad6c48-703c-460e-b533-ce410ecc7ccf None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:59.756771 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:59.774080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5682ce-0ce9-429d-83a2-3152195b7690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:59.774672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5682ce-0ce9-429d-83a2-3152195b7690 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:59.774947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5682ce-0ce9-429d-83a2-3152195b7690 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:59.775137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 263/516] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:56:59.775290 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5682ce-0ce9-429d-83a2-3152195b7690 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:59.775985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ce1ce-ef11-4ef5-bd38-a0b5b959f680 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.776076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5682ce-0ce9-429d-83a2-3152195b7690 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:59.778198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5682ce-0ce9-429d-83a2-3152195b7690 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:59.799677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 254/517] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:56:59 2022] POST /identity/v3/users => generated 247 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:56:59.800682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5682ce-0ce9-429d-83a2-3152195b7690 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.829295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:56:59.890919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ce1ce-ef11-4ef5-bd38-a0b5b959f680 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:59.921398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5682ce-0ce9-429d-83a2-3152195b7690 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:56:59.930180 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6695d5-e3e5-4e93-b6a7-2c29a1cd6362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:59.930800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6695d5-e3e5-4e93-b6a7-2c29a1cd6362 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:59.931105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6695d5-e3e5-4e93-b6a7-2c29a1cd6362 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:59.931345 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6695d5-e3e5-4e93-b6a7-2c29a1cd6362 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:59.931993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6695d5-e3e5-4e93-b6a7-2c29a1cd6362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:59.932292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6695d5-e3e5-4e93-b6a7-2c29a1cd6362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:59.934766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6695d5-e3e5-4e93-b6a7-2c29a1cd6362 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:59.947513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 264/518] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:59.948437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6695d5-e3e5-4e93-b6a7-2c29a1cd6362 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:56:59.967502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596d704-bdfa-4bb4-8e31-94102818982c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:56:59.968127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596d704-bdfa-4bb4-8e31-94102818982c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:56:59.968403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596d704-bdfa-4bb4-8e31-94102818982c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:56:59.968678 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6596d704-bdfa-4bb4-8e31-94102818982c None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:56:59.969352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596d704-bdfa-4bb4-8e31-94102818982c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:56:59.969657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596d704-bdfa-4bb4-8e31-94102818982c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:56:59.971041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6596d704-bdfa-4bb4-8e31-94102818982c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:56:59.985488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 255/519] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:56:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:56:59.986319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596d704-bdfa-4bb4-8e31-94102818982c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:00.025667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:00.077374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6695d5-e3e5-4e93-b6a7-2c29a1cd6362 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:00.081358 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:00.105340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6596d704-bdfa-4bb4-8e31-94102818982c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:00.115899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac1a4a-a834-4631-83c4-588f9110214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:00.116501 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac1a4a-a834-4631-83c4-588f9110214f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:00.116777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac1a4a-a834-4631-83c4-588f9110214f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:00.117047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac1a4a-a834-4631-83c4-588f9110214f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:00.141284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c19600-f35c-4cc4-805b-0442429d75b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:00.141910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c19600-f35c-4cc4-805b-0442429d75b1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:00.142188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c19600-f35c-4cc4-805b-0442429d75b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:00.142455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c19600-f35c-4cc4-805b-0442429d75b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:00.153580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c19600-f35c-4cc4-805b-0442429d75b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:00.155020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c19600-f35c-4cc4-805b-0442429d75b1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-c9c19600-f35c-4cc4-805b-0442429d75b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:00.169663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:00.170589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:00.176303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:00.176303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:00.176303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:00.176303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 256/520] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:56:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:00.176303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c19600-f35c-4cc4-805b-0442429d75b1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:00.188588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac1a4a-a834-4631-83c4-588f9110214f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:00.190067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edac1a4a-a834-4631-83c4-588f9110214f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:00.264074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:00.280683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c19600-f35c-4cc4-805b-0442429d75b1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:00.298756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 265/521] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:56:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:57:00.299438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac1a4a-a834-4631-83c4-588f9110214f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:00.305846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:00.317684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9712d4c-8ace-402f-a37f-7074118f7db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:00.318306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712d4c-8ace-402f-a37f-7074118f7db7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:00.318583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712d4c-8ace-402f-a37f-7074118f7db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:00.318851 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9712d4c-8ace-402f-a37f-7074118f7db7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:00.326295 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9712d4c-8ace-402f-a37f-7074118f7db7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:00.327741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9712d4c-8ace-402f-a37f-7074118f7db7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-d9712d4c-8ace-402f-a37f-7074118f7db7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:00.344350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:00.345289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:00.351329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:00.351329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:00.351329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:00.351329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:00.351329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:00.351329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:00.351329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:00.351329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:00.351329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 257/522] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:00.351329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9712d4c-8ace-402f-a37f-7074118f7db7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:00.413785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac1a4a-a834-4631-83c4-588f9110214f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:00.451525 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08096df3-50a4-4ea8-826c-173b43f64b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:00.452155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9712d4c-8ace-402f-a37f-7074118f7db7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:00.452276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08096df3-50a4-4ea8-826c-173b43f64b64 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:00.452432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08096df3-50a4-4ea8-826c-173b43f64b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:00.452690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08096df3-50a4-4ea8-826c-173b43f64b64 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:00.460361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08096df3-50a4-4ea8-826c-173b43f64b64 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:00.462038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08096df3-50a4-4ea8-826c-173b43f64b64 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-08096df3-50a4-4ea8-826c-173b43f64b64 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:00.474555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:00.475520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 266/523] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:00 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:00.476405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08096df3-50a4-4ea8-826c-173b43f64b64 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:00.488682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c083c-f746-489a-9068-3d5b3f8227ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:00.489337 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c083c-f746-489a-9068-3d5b3f8227ae None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:00.489617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c083c-f746-489a-9068-3d5b3f8227ae None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:00.489889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c083c-f746-489a-9068-3d5b3f8227ae None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:00.490526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c083c-f746-489a-9068-3d5b3f8227ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:00.490828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c083c-f746-489a-9068-3d5b3f8227ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:00.492204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c083c-f746-489a-9068-3d5b3f8227ae None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:00.509075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 258/524] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:00.509958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c083c-f746-489a-9068-3d5b3f8227ae None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:00.520885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:00.546037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:00.588692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08096df3-50a4-4ea8-826c-173b43f64b64 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:00.612634 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c083c-f746-489a-9068-3d5b3f8227ae None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:00.627694 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bdb62-8cf1-41ef-a7bf-46cbd8ec712d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:00.628326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bdb62-8cf1-41ef-a7bf-46cbd8ec712d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:00.628604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bdb62-8cf1-41ef-a7bf-46cbd8ec712d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:00.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331bdb62-8cf1-41ef-a7bf-46cbd8ec712d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:00.641607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bdb62-8cf1-41ef-a7bf-46cbd8ec712d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:00.643218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331bdb62-8cf1-41ef-a7bf-46cbd8ec712d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:00.647999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c89336-c8d4-44d7-b984-51626f8cef40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:00.648596 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c89336-c8d4-44d7-b984-51626f8cef40 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:00.648871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c89336-c8d4-44d7-b984-51626f8cef40 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:00.649164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c89336-c8d4-44d7-b984-51626f8cef40 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:00.649847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c89336-c8d4-44d7-b984-51626f8cef40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:00.650146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c89336-c8d4-44d7-b984-51626f8cef40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:00.651601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c89336-c8d4-44d7-b984-51626f8cef40 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-331bdb62-8cf1-41ef-a7bf-46cbd8ec712d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:00.658609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:00.659705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:00.660756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:00.660756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:00.660756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:00.662283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 267/525] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:00.663184 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bdb62-8cf1-41ef-a7bf-46cbd8ec712d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:00.670533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 259/526] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:00.671222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c89336-c8d4-44d7-b984-51626f8cef40 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:00.756625 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:00.789839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331bdb62-8cf1-41ef-a7bf-46cbd8ec712d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:00.796849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c89336-c8d4-44d7-b984-51626f8cef40 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:00.809310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:00.829937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffc85d-fdda-4590-9838-af971c88f197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:00.830831 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffc85d-fdda-4590-9838-af971c88f197 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:00.831321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffc85d-fdda-4590-9838-af971c88f197 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:00.831788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ffc85d-fdda-4590-9838-af971c88f197 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:00.832649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffc85d-fdda-4590-9838-af971c88f197 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:00.833096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffc85d-fdda-4590-9838-af971c88f197 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:00.833399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add574a-51a2-4b8f-805e-e4aeccd3b194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:00.834022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add574a-51a2-4b8f-805e-e4aeccd3b194 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:00.834301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add574a-51a2-4b8f-805e-e4aeccd3b194 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:00.834575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add574a-51a2-4b8f-805e-e4aeccd3b194 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:00.835331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ffc85d-fdda-4590-9838-af971c88f197 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:00.843115 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add574a-51a2-4b8f-805e-e4aeccd3b194 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:00.844785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add574a-51a2-4b8f-805e-e4aeccd3b194 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-2add574a-51a2-4b8f-805e-e4aeccd3b194 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:00.856081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:00.857121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:00.862700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:00.862700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:00.862700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:00.862700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:00.862700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:00.862700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:00.862700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:00.862700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:00.862700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 260/527] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:00.862700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add574a-51a2-4b8f-805e-e4aeccd3b194 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:00.878562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 268/528] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 22 21:57:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:00.880255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffc85d-fdda-4590-9838-af971c88f197 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:00.966220 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add574a-51a2-4b8f-805e-e4aeccd3b194 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:00.991836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ffc85d-fdda-4590-9838-af971c88f197 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:00.993430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:01.004576 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fd76b-5d89-4835-a92d-17e37d4ec0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:01.005208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fd76b-5d89-4835-a92d-17e37d4ec0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:01.005621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fd76b-5d89-4835-a92d-17e37d4ec0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:01.005794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814fd76b-5d89-4835-a92d-17e37d4ec0f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:01.006624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fd76b-5d89-4835-a92d-17e37d4ec0f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:01.006731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fd76b-5d89-4835-a92d-17e37d4ec0f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:01.008099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814fd76b-5d89-4835-a92d-17e37d4ec0f6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:01.024935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 261/529] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:01.024935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fd76b-5d89-4835-a92d-17e37d4ec0f6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:01.029042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1d8ae-ad70-49d5-ad30-d479fdd75007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:01.029670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1d8ae-ad70-49d5-ad30-d479fdd75007 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:01.029952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1d8ae-ad70-49d5-ad30-d479fdd75007 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:01.030224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb1d8ae-ad70-49d5-ad30-d479fdd75007 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:01.030951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1d8ae-ad70-49d5-ad30-d479fdd75007 None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=195565a7695b4d9b82e0ee76181fee2f) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:01.031260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1d8ae-ad70-49d5-ad30-d479fdd75007 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:01.032688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb1d8ae-ad70-49d5-ad30-d479fdd75007 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:01.038162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 269/530] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:00 2022] GET /identity/v3/role_assignments?user.id=195565a7695b4d9b82e0ee76181fee2f&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:01.038914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1d8ae-ad70-49d5-ad30-d479fdd75007 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:01.089332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:01.176650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb1d8ae-ad70-49d5-ad30-d479fdd75007 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:01.189582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814fd76b-5d89-4835-a92d-17e37d4ec0f6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:01.220117 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be19c-68ec-4343-adde-09ef99d6e01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:01.220880 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be19c-68ec-4343-adde-09ef99d6e01c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:01.222084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be19c-68ec-4343-adde-09ef99d6e01c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:01.222369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0be19c-68ec-4343-adde-09ef99d6e01c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:01.229967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e0d2c-1136-42c8-850a-354f1d97284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:01.230988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0d2c-1136-42c8-850a-354f1d97284d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:01.231465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0d2c-1136-42c8-850a-354f1d97284d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:01.231929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e0d2c-1136-42c8-850a-354f1d97284d None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:01.232885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e0d2c-1136-42c8-850a-354f1d97284d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:01.233437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e0d2c-1136-42c8-850a-354f1d97284d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:01.235153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e0d2c-1136-42c8-850a-354f1d97284d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:01.251890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 262/531] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:01.253042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e0d2c-1136-42c8-850a-354f1d97284d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:01.308557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0be19c-68ec-4343-adde-09ef99d6e01c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:01.309675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:01.312485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0be19c-68ec-4343-adde-09ef99d6e01c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:01.367461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e0d2c-1136-42c8-850a-354f1d97284d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:01.373481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:01.406193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd7ff5-ad24-43e6-99f8-0b86f75a79c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:01.407472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd7ff5-ad24-43e6-99f8-0b86f75a79c4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:01.407946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd7ff5-ad24-43e6-99f8-0b86f75a79c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:01.408430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd7ff5-ad24-43e6-99f8-0b86f75a79c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:01.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd7ff5-ad24-43e6-99f8-0b86f75a79c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:01.418475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dd7ff5-ad24-43e6-99f8-0b86f75a79c4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-93dd7ff5-ad24-43e6-99f8-0b86f75a79c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:01.431303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:01.432169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:01.433552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:01.433552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:01.433552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:01.433552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:01.433552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:01.433552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:01.433552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:01.433552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:01.436083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 263/532] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:01.437203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd7ff5-ad24-43e6-99f8-0b86f75a79c4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:01.437409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 270/533] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:57:01.443636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be19c-68ec-4343-adde-09ef99d6e01c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:01.551809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd7ff5-ad24-43e6-99f8-0b86f75a79c4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:01.560458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:01.577681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0be19c-68ec-4343-adde-09ef99d6e01c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:01.590501 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf409b-2522-47f4-91c7-f521fe7c7673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:01.591721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf409b-2522-47f4-91c7-f521fe7c7673 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:01.592195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf409b-2522-47f4-91c7-f521fe7c7673 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:01.592663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf409b-2522-47f4-91c7-f521fe7c7673 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:01.593619 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf409b-2522-47f4-91c7-f521fe7c7673 None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=d011e54e12674eaeb82c68da628ecb9d) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:01.594143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf409b-2522-47f4-91c7-f521fe7c7673 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:01.595770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf409b-2522-47f4-91c7-f521fe7c7673 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:01.602053 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 264/534] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:01 2022] GET /identity/v3/role_assignments?user.id=d011e54e12674eaeb82c68da628ecb9d&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:01.603083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf409b-2522-47f4-91c7-f521fe7c7673 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:01.639003 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959a2d2-6867-4c8c-9a30-23cd42d91e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:01.640982 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959a2d2-6867-4c8c-9a30-23cd42d91e9d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:01.641552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959a2d2-6867-4c8c-9a30-23cd42d91e9d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:01.642025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f959a2d2-6867-4c8c-9a30-23cd42d91e9d None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:01.642949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959a2d2-6867-4c8c-9a30-23cd42d91e9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:01.643533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959a2d2-6867-4c8c-9a30-23cd42d91e9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:01.645272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f959a2d2-6867-4c8c-9a30-23cd42d91e9d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:01.659660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:01.670121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 271/535] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:01.676241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959a2d2-6867-4c8c-9a30-23cd42d91e9d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:01.799696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf409b-2522-47f4-91c7-f521fe7c7673 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:01.833833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f959a2d2-6867-4c8c-9a30-23cd42d91e9d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:01.839304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0005d-c5ba-471e-b78a-0f67012409ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:01.840258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0005d-c5ba-471e-b78a-0f67012409ad None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:01.840732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0005d-c5ba-471e-b78a-0f67012409ad None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:01.841502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0005d-c5ba-471e-b78a-0f67012409ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:01.853281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0005d-c5ba-471e-b78a-0f67012409ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:01.856124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0005d-c5ba-471e-b78a-0f67012409ad None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-4bb0005d-c5ba-471e-b78a-0f67012409ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:01.878538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:01.879485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:01.880303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:01.880303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:01.880303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:01.880303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 265/536] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:01.885159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35182f08-c0e8-4096-8bdf-0a5a176e21ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:01.885159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35182f08-c0e8-4096-8bdf-0a5a176e21ee None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:01.885159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35182f08-c0e8-4096-8bdf-0a5a176e21ee None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:01.885159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35182f08-c0e8-4096-8bdf-0a5a176e21ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:01.885501 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35182f08-c0e8-4096-8bdf-0a5a176e21ee None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=fcfdbb6b3dd04478ae99681cc649fdb9) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:01.885501 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35182f08-c0e8-4096-8bdf-0a5a176e21ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:01.885676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35182f08-c0e8-4096-8bdf-0a5a176e21ee None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:01.888201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0005d-c5ba-471e-b78a-0f67012409ad None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:01.897503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 272/537] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:01 2022] GET /identity/v3/role_assignments?user.id=fcfdbb6b3dd04478ae99681cc649fdb9&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:01.922317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:02.026636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0005d-c5ba-471e-b78a-0f67012409ad None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:02.072102 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a8c3e-a895-4989-a35b-1f046b41c292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:02.072734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a8c3e-a895-4989-a35b-1f046b41c292 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:02.073010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a8c3e-a895-4989-a35b-1f046b41c292 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:02.073528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96a8c3e-a895-4989-a35b-1f046b41c292 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:02.074351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a8c3e-a895-4989-a35b-1f046b41c292 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:02.074656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a8c3e-a895-4989-a35b-1f046b41c292 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:02.076152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96a8c3e-a895-4989-a35b-1f046b41c292 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:02.098630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 266/538] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:02.100280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecda99-f11b-405a-a325-bfb4580f3437 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:02.100566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecda99-f11b-405a-a325-bfb4580f3437 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:02.100838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecda99-f11b-405a-a325-bfb4580f3437 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:02.104036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 273/539] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:57:02.105854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a8c3e-a895-4989-a35b-1f046b41c292 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:02.109227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203a9af-6202-4e82-ba82-6e0a3138730b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:02.109511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203a9af-6202-4e82-ba82-6e0a3138730b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:02.109783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203a9af-6202-4e82-ba82-6e0a3138730b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:02.134819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9203a9af-6202-4e82-ba82-6e0a3138730b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:02.136117 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9203a9af-6202-4e82-ba82-6e0a3138730b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:02.222053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:02.250301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 274/540] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:57:02.258414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:02.285860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96a8c3e-a895-4989-a35b-1f046b41c292 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:02.332428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c503b20-f334-44ec-a70a-353b81e793de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:02.332972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c503b20-f334-44ec-a70a-353b81e793de None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:02.333274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c503b20-f334-44ec-a70a-353b81e793de None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:02.333544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c503b20-f334-44ec-a70a-353b81e793de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:02.345831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c503b20-f334-44ec-a70a-353b81e793de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:02.347303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c503b20-f334-44ec-a70a-353b81e793de None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-2c503b20-f334-44ec-a70a-353b81e793de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:02.359141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:02.360010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:02.360830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:02.360830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:02.360830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:02.360830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:02.360830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:02.360830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:02.360830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:02.360830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:02.364691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 267/541] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:02.365935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203a9af-6202-4e82-ba82-6e0a3138730b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:02.484669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203a9af-6202-4e82-ba82-6e0a3138730b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:02.524533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd0d56-110a-49d3-8ff0-64900fba820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:02.525153 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd0d56-110a-49d3-8ff0-64900fba820b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:02.525431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd0d56-110a-49d3-8ff0-64900fba820b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:02.525697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afd0d56-110a-49d3-8ff0-64900fba820b None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:02.527011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd0d56-110a-49d3-8ff0-64900fba820b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:02.527298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd0d56-110a-49d3-8ff0-64900fba820b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:02.528791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afd0d56-110a-49d3-8ff0-64900fba820b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:02.541122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 275/542] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:02.547006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c503b20-f334-44ec-a70a-353b81e793de None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:02.655398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c503b20-f334-44ec-a70a-353b81e793de None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:02.673787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd0d56-110a-49d3-8ff0-64900fba820b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:02.696805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4e98e-1f29-4bd4-8ca3-2ba937051e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:02.700079 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4e98e-1f29-4bd4-8ca3-2ba937051e0d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:02.702051 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4e98e-1f29-4bd4-8ca3-2ba937051e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:02.703363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4e98e-1f29-4bd4-8ca3-2ba937051e0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:02.705949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4e98e-1f29-4bd4-8ca3-2ba937051e0d None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=18c956fa035f40fa88fbf662541121ab) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:02.706467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4e98e-1f29-4bd4-8ca3-2ba937051e0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:02.709138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4e98e-1f29-4bd4-8ca3-2ba937051e0d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:02.725886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 268/543] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:02 2022] GET /identity/v3/role_assignments?user.id=18c956fa035f40fa88fbf662541121ab&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:02.760616 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:02.825979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afd0d56-110a-49d3-8ff0-64900fba820b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:02.876449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35216d49-d872-4a0e-a2fd-3b02b14f67e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:02.877494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35216d49-d872-4a0e-a2fd-3b02b14f67e7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:02.877997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35216d49-d872-4a0e-a2fd-3b02b14f67e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:02.878464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35216d49-d872-4a0e-a2fd-3b02b14f67e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:02.902571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35216d49-d872-4a0e-a2fd-3b02b14f67e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-35216d49-d872-4a0e-a2fd-3b02b14f67e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:02.908524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:02.909446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:02.910232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:02.912993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 276/544] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:02.919326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4e98e-1f29-4bd4-8ca3-2ba937051e0d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:02.930685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:03.041612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4e98e-1f29-4bd4-8ca3-2ba937051e0d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:03.104805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ae779-a760-469b-9348-219f7fb3d408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:03.105735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ae779-a760-469b-9348-219f7fb3d408 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:03.106019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ae779-a760-469b-9348-219f7fb3d408 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:03.106290 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ae779-a760-469b-9348-219f7fb3d408 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:03.106954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ae779-a760-469b-9348-219f7fb3d408 None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:03.107256 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ae779-a760-469b-9348-219f7fb3d408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:03.108628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ae779-a760-469b-9348-219f7fb3d408 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:03.121202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 269/545] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:03.127430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35216d49-d872-4a0e-a2fd-3b02b14f67e7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:03.312279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35216d49-d872-4a0e-a2fd-3b02b14f67e7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:03.329289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:03.397829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eeb48f-bb50-4fa2-a0b6-93cdbfda787c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:03.398473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeb48f-bb50-4fa2-a0b6-93cdbfda787c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:03.398752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeb48f-bb50-4fa2-a0b6-93cdbfda787c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:03.399019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eeb48f-bb50-4fa2-a0b6-93cdbfda787c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:03.416953 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eeb48f-bb50-4fa2-a0b6-93cdbfda787c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:03.418443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eeb48f-bb50-4fa2-a0b6-93cdbfda787c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-28eeb48f-bb50-4fa2-a0b6-93cdbfda787c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:03.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:03.444379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:03.445244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:03.445244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:03.445244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:03.447004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 277/546] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:03.474268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ae779-a760-469b-9348-219f7fb3d408 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:03.624416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:03.701559 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed484153-fc10-4ea1-a776-71682ec4bb80 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:03.702528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed484153-fc10-4ea1-a776-71682ec4bb80 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:03.703105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed484153-fc10-4ea1-a776-71682ec4bb80 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:03.704980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 278/547] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:57:03.715324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ae779-a760-469b-9348-219f7fb3d408 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:03.722952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bca42d-7aaf-4a79-8459-6ac98819a4d6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:03.725312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bca42d-7aaf-4a79-8459-6ac98819a4d6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:03.725875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bca42d-7aaf-4a79-8459-6ac98819a4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:03.773914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-03bca42d-7aaf-4a79-8459-6ac98819a4d6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:03.775111 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-03bca42d-7aaf-4a79-8459-6ac98819a4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:03.807457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f96f33-f56c-464b-8681-12ead4ca6504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:03.808099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f96f33-f56c-464b-8681-12ead4ca6504 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:03.808527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f96f33-f56c-464b-8681-12ead4ca6504 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:03.808798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f96f33-f56c-464b-8681-12ead4ca6504 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:03.809465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f96f33-f56c-464b-8681-12ead4ca6504 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:03.809764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f96f33-f56c-464b-8681-12ead4ca6504 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:03.811119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f96f33-f56c-464b-8681-12ead4ca6504 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:03.851423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 270/548] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:03.857856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f96f33-f56c-464b-8681-12ead4ca6504 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:03.902723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 279/549] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:04.002961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f96f33-f56c-464b-8681-12ead4ca6504 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:04.049471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:04.060046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b758db-7b9b-4216-828f-6ac7e1700bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:04.060689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b758db-7b9b-4216-828f-6ac7e1700bfb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:04.060969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b758db-7b9b-4216-828f-6ac7e1700bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:04.061270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b758db-7b9b-4216-828f-6ac7e1700bfb None admin] PATH_INFO: `/v3/users/swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:04.072019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b758db-7b9b-4216-828f-6ac7e1700bfb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:04.074468 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b758db-7b9b-4216-828f-6ac7e1700bfb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-64b758db-7b9b-4216-828f-6ac7e1700bfb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:04.095399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:04.096597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:04.097545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:04.097545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:04.097545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:04.097545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:04.097545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:04.097545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:04.097545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:04.097545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:04.097545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:04.097545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:04.100090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 271/550] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:04.106193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bca42d-7aaf-4a79-8459-6ac98819a4d6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:04.208857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:04.217374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bca42d-7aaf-4a79-8459-6ac98819a4d6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:04.242049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b758db-7b9b-4216-828f-6ac7e1700bfb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:04.254170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555335b-a812-4a9c-92e8-02c633a75cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:04.255194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555335b-a812-4a9c-92e8-02c633a75cff None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:04.255696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555335b-a812-4a9c-92e8-02c633a75cff None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:04.256283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555335b-a812-4a9c-92e8-02c633a75cff None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:04.257019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555335b-a812-4a9c-92e8-02c633a75cff None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:04.257354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555335b-a812-4a9c-92e8-02c633a75cff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:04.258777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555335b-a812-4a9c-92e8-02c633a75cff None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:04.291911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 280/551] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 22 21:57:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:04.294688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555335b-a812-4a9c-92e8-02c633a75cff None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:04.438887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b758db-7b9b-4216-828f-6ac7e1700bfb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:04.489379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555335b-a812-4a9c-92e8-02c633a75cff None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:04.513758 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc198d1-9a69-4f2d-9437-2ad6863e3f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:04.514407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc198d1-9a69-4f2d-9437-2ad6863e3f6a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:04.514679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc198d1-9a69-4f2d-9437-2ad6863e3f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:04.514944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc198d1-9a69-4f2d-9437-2ad6863e3f6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:04.543067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc198d1-9a69-4f2d-9437-2ad6863e3f6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0fc198d1-9a69-4f2d-9437-2ad6863e3f6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:04.548369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:04.549488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:04.550317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 272/552] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:04.559799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc198d1-9a69-4f2d-9437-2ad6863e3f6a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:04.561861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c65604-adca-4d7f-81e6-81b4dec96e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:04.562798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c65604-adca-4d7f-81e6-81b4dec96e95 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:04.563203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c65604-adca-4d7f-81e6-81b4dec96e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:04.563651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c65604-adca-4d7f-81e6-81b4dec96e95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:04.590694 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:04.629632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:04.649927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c65604-adca-4d7f-81e6-81b4dec96e95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:04.651531 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c65604-adca-4d7f-81e6-81b4dec96e95 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:04.684583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc198d1-9a69-4f2d-9437-2ad6863e3f6a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:04.719124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7b1adf-e33b-4396-b352-9f58563299e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:04.720040 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b1adf-e33b-4396-b352-9f58563299e7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:04.720573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b1adf-e33b-4396-b352-9f58563299e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:04.720839 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7b1adf-e33b-4396-b352-9f58563299e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:04.721518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b1adf-e33b-4396-b352-9f58563299e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:04.721816 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b1adf-e33b-4396-b352-9f58563299e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:04.723173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7b1adf-e33b-4396-b352-9f58563299e7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:04.736785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 273/553] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:04.736785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23532-dcd6-46d4-bcd9-1b2f878d068f None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:04.736785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23532-dcd6-46d4-bcd9-1b2f878d068f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:04.736785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b23532-dcd6-46d4-bcd9-1b2f878d068f None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:04.736785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 274/554] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:04.736785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f41f6-8163-4c2c-a710-88a6deb4d3ec None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:04.737370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f41f6-8163-4c2c-a710-88a6deb4d3ec None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:04.737462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f41f6-8163-4c2c-a710-88a6deb4d3ec None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:04.738762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 275/555] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:04.742924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f41f6-8163-4c2c-a710-88a6deb4d3ec None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:04.759867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 281/556] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:57:04.760982 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22577750-d659-4ef9-b99f-e1af2ea9e4a8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:04.761297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22577750-d659-4ef9-b99f-e1af2ea9e4a8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:04.761570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22577750-d659-4ef9-b99f-e1af2ea9e4a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:04.778522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-22577750-d659-4ef9-b99f-e1af2ea9e4a8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:04.779702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-22577750-d659-4ef9-b99f-e1af2ea9e4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:04.827005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:04.849072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 282/557] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:04.850298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932eabd-e425-4354-b1ed-11c3f2431348 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:04.850599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932eabd-e425-4354-b1ed-11c3f2431348 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:04.850857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932eabd-e425-4354-b1ed-11c3f2431348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:04.869739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7932eabd-e425-4354-b1ed-11c3f2431348 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:04.872316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7932eabd-e425-4354-b1ed-11c3f2431348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:04.873201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f41f6-8163-4c2c-a710-88a6deb4d3ec None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:04.884500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:04.911779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770703f-cc54-40f9-8b6b-c0823bb19458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:04.912448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770703f-cc54-40f9-8b6b-c0823bb19458 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:04.912732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770703f-cc54-40f9-8b6b-c0823bb19458 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:04.913005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7770703f-cc54-40f9-8b6b-c0823bb19458 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:04.924455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770703f-cc54-40f9-8b6b-c0823bb19458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:04.926106 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7770703f-cc54-40f9-8b6b-c0823bb19458 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-7770703f-cc54-40f9-8b6b-c0823bb19458 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:04.943401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:04.944370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:04.945336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:04.945336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:04.945336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:04.947505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 276/558] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:04.949614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770703f-cc54-40f9-8b6b-c0823bb19458 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:04.953262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 283/559] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:04.954438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932eabd-e425-4354-b1ed-11c3f2431348 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:05.102157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7770703f-cc54-40f9-8b6b-c0823bb19458 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:05.121330 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932eabd-e425-4354-b1ed-11c3f2431348 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:05.146751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:05.170657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa220c9-66d7-481a-803d-154d3ef25c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:05.171340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa220c9-66d7-481a-803d-154d3ef25c3f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.171623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa220c9-66d7-481a-803d-154d3ef25c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.172548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa220c9-66d7-481a-803d-154d3ef25c3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.186469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0c186-077b-499f-bf59-0590762c8ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:05.188918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c186-077b-499f-bf59-0590762c8ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.189233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c186-077b-499f-bf59-0590762c8ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.189509 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd0c186-077b-499f-bf59-0590762c8ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.191976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0c186-077b-499f-bf59-0590762c8ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:05.192294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0c186-077b-499f-bf59-0590762c8ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:05.194709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd0c186-077b-499f-bf59-0590762c8ea5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:05.196980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa220c9-66d7-481a-803d-154d3ef25c3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:05.198937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa220c9-66d7-481a-803d-154d3ef25c3f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:05.212122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 284/560] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:05.213043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0c186-077b-499f-bf59-0590762c8ea5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-9fa220c9-66d7-481a-803d-154d3ef25c3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:05.219930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:05.220872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:05.221808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:05.221808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:05.221808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:05.223698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 277/561] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:05.224674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa220c9-66d7-481a-803d-154d3ef25c3f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:05.246376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:05.340546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa220c9-66d7-481a-803d-154d3ef25c3f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:05.347233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd0c186-077b-499f-bf59-0590762c8ea5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:05.375615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac20fe-09b4-46b2-b964-e7392396334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:05.376235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac20fe-09b4-46b2-b964-e7392396334a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.376515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac20fe-09b4-46b2-b964-e7392396334a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.376783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ac20fe-09b4-46b2-b964-e7392396334a None admin] PATH_INFO: `/v3/users/glance` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.384126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac20fe-09b4-46b2-b964-e7392396334a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:05.385718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ac20fe-09b4-46b2-b964-e7392396334a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:05.386690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dd2b0-db5a-4a3d-80ed-c45c2c0b9596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:05.387736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dd2b0-db5a-4a3d-80ed-c45c2c0b9596 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.388756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dd2b0-db5a-4a3d-80ed-c45c2c0b9596 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.389200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9dd2b0-db5a-4a3d-80ed-c45c2c0b9596 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-c8ac20fe-09b4-46b2-b964-e7392396334a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:05.396964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:05.398110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 278/562] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:57:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:05.399083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac20fe-09b4-46b2-b964-e7392396334a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:05.404106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9dd2b0-db5a-4a3d-80ed-c45c2c0b9596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bc9dd2b0-db5a-4a3d-80ed-c45c2c0b9596 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:05.409919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:05.411085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:05.412085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:05.414476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 285/563] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:05.415438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dd2b0-db5a-4a3d-80ed-c45c2c0b9596 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:05.452919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:05.481959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:05.506367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ac20fe-09b4-46b2-b964-e7392396334a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:05.521038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9dd2b0-db5a-4a3d-80ed-c45c2c0b9596 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:05.538237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9bf48-4a37-4b74-9ed1-4a29bd41a7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:05.538830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9bf48-4a37-4b74-9ed1-4a29bd41a7bc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.539103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9bf48-4a37-4b74-9ed1-4a29bd41a7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.539369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9bf48-4a37-4b74-9ed1-4a29bd41a7bc None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.539991 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9bf48-4a37-4b74-9ed1-4a29bd41a7bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:05.540293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9bf48-4a37-4b74-9ed1-4a29bd41a7bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:05.542234 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9bf48-4a37-4b74-9ed1-4a29bd41a7bc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:05.553863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 279/564] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:05.554639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9bf48-4a37-4b74-9ed1-4a29bd41a7bc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:05.559090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdbaec-62c0-4712-87f9-c0cceb839929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:05.559710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdbaec-62c0-4712-87f9-c0cceb839929 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.559982 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdbaec-62c0-4712-87f9-c0cceb839929 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.560246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdbaec-62c0-4712-87f9-c0cceb839929 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.570359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdbaec-62c0-4712-87f9-c0cceb839929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9acdbaec-62c0-4712-87f9-c0cceb839929 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:05.576316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:05.577398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 286/565] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:05.578242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdbaec-62c0-4712-87f9-c0cceb839929 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:05.656770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9bf48-4a37-4b74-9ed1-4a29bd41a7bc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:05.670907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:05.679644 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdbaec-62c0-4712-87f9-c0cceb839929 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:05.690756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac65e8-9f96-42cc-b70d-20486d122cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:05.691356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac65e8-9f96-42cc-b70d-20486d122cff None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.691633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac65e8-9f96-42cc-b70d-20486d122cff None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.691903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac65e8-9f96-42cc-b70d-20486d122cff None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.692618 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac65e8-9f96-42cc-b70d-20486d122cff None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:05.692929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac65e8-9f96-42cc-b70d-20486d122cff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:05.694370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac65e8-9f96-42cc-b70d-20486d122cff None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:05.695952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:05.712498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a883f11-9840-4649-984b-5d4ae3ae8ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:05.712888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 280/566] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 22 21:57:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:05.713158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a883f11-9840-4649-984b-5d4ae3ae8ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.713928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a883f11-9840-4649-984b-5d4ae3ae8ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.714216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a883f11-9840-4649-984b-5d4ae3ae8ca8 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.714783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfd440-1cd6-48cb-adaa-42b00c7a9d8f None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.714892 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a883f11-9840-4649-984b-5d4ae3ae8ca8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:05.715063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfd440-1cd6-48cb-adaa-42b00c7a9d8f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.715158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a883f11-9840-4649-984b-5d4ae3ae8ca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:05.715333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfd440-1cd6-48cb-adaa-42b00c7a9d8f None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.716942 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a883f11-9840-4649-984b-5d4ae3ae8ca8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:05.717063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 281/567] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:05.717552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cfd440-1cd6-48cb-adaa-42b00c7a9d8f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:05.729019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 287/568] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:05.730285 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a883f11-9840-4649-984b-5d4ae3ae8ca8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:05.817975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cfd440-1cd6-48cb-adaa-42b00c7a9d8f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:05.828151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a883f11-9840-4649-984b-5d4ae3ae8ca8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:05.851453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49aa3a-c66c-4560-87d4-75e252782132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:05.852070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49aa3a-c66c-4560-87d4-75e252782132 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.852345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49aa3a-c66c-4560-87d4-75e252782132 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.852614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49aa3a-c66c-4560-87d4-75e252782132 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.859656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49aa3a-c66c-4560-87d4-75e252782132 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:05.861094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49aa3a-c66c-4560-87d4-75e252782132 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:05.861480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd8fd-b0dc-43d2-a5e1-dff0782cf1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:05.862074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd8fd-b0dc-43d2-a5e1-dff0782cf1fd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.862347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd8fd-b0dc-43d2-a5e1-dff0782cf1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.862615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffd8fd-b0dc-43d2-a5e1-dff0782cf1fd None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.863256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffd8fd-b0dc-43d2-a5e1-dff0782cf1fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:05.863553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffd8fd-b0dc-43d2-a5e1-dff0782cf1fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:05.864879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffd8fd-b0dc-43d2-a5e1-dff0782cf1fd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-3f49aa3a-c66c-4560-87d4-75e252782132 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:05.872730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:05.873665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:05.874577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:05.874577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:05.874577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:05.874577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:05.874577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:05.874577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:05.874577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:05.874577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:05.875016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 288/569] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:05.875016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd8fd-b0dc-43d2-a5e1-dff0782cf1fd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:05.875151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 282/570] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:05.875222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ddb49-cd49-44c3-8983-f58242ee8963 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:05.875623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ddb49-cd49-44c3-8983-f58242ee8963 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:05.875756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535ddb49-cd49-44c3-8983-f58242ee8963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:05.892954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-535ddb49-cd49-44c3-8983-f58242ee8963 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:05.894182 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-535ddb49-cd49-44c3-8983-f58242ee8963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:05.895154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:05.942773 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:05.964668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 283/571] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:05.965554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ddb49-cd49-44c3-8983-f58242ee8963 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:05.977375 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffd8fd-b0dc-43d2-a5e1-dff0782cf1fd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.013021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2cbc-addf-4e3e-91aa-4860b51240a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.013769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2cbc-addf-4e3e-91aa-4860b51240a6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.014111 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2cbc-addf-4e3e-91aa-4860b51240a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.014338 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d2cbc-addf-4e3e-91aa-4860b51240a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.025024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d2cbc-addf-4e3e-91aa-4860b51240a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:06.027855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d2cbc-addf-4e3e-91aa-4860b51240a6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-da4d2cbc-addf-4e3e-91aa-4860b51240a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:06.042361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:06.043288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:06.044217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:06.044217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:06.044217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:06.044217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 289/572] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:06.044217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2cbc-addf-4e3e-91aa-4860b51240a6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.093870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535ddb49-cd49-44c3-8983-f58242ee8963 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.126496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:06.130516 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685c780-c640-47f1-b6cb-46b1cb80cc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.131174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685c780-c640-47f1-b6cb-46b1cb80cc69 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.131429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685c780-c640-47f1-b6cb-46b1cb80cc69 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.131703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1685c780-c640-47f1-b6cb-46b1cb80cc69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.142985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685c780-c640-47f1-b6cb-46b1cb80cc69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:06.144562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1685c780-c640-47f1-b6cb-46b1cb80cc69 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-1685c780-c640-47f1-b6cb-46b1cb80cc69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:06.161074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:06.162026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:06.162919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:06.162919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:06.162919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:06.162919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 284/573] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:06.163140 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685c780-c640-47f1-b6cb-46b1cb80cc69 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.169331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d2cbc-addf-4e3e-91aa-4860b51240a6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.201143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:06.205263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a40ca5-513e-4187-8237-38a3091aad08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.205899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a40ca5-513e-4187-8237-38a3091aad08 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.206178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a40ca5-513e-4187-8237-38a3091aad08 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.206452 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a40ca5-513e-4187-8237-38a3091aad08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.218216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a40ca5-513e-4187-8237-38a3091aad08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:06.219672 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a40ca5-513e-4187-8237-38a3091aad08 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b6a40ca5-513e-4187-8237-38a3091aad08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:06.234781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:06.235776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:06.236648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:06.236648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:06.236648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:06.236648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 290/574] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:06.236648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a40ca5-513e-4187-8237-38a3091aad08 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.275684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1685c780-c640-47f1-b6cb-46b1cb80cc69 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.310614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09794f09-c67e-4dde-9dc6-b701e3e33585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.311214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09794f09-c67e-4dde-9dc6-b701e3e33585 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.311495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09794f09-c67e-4dde-9dc6-b701e3e33585 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.311766 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09794f09-c67e-4dde-9dc6-b701e3e33585 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.312447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09794f09-c67e-4dde-9dc6-b701e3e33585 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:06.312750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09794f09-c67e-4dde-9dc6-b701e3e33585 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:06.314209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09794f09-c67e-4dde-9dc6-b701e3e33585 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:06.325783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 285/575] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:06.326375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09794f09-c67e-4dde-9dc6-b701e3e33585 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.341570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a40ca5-513e-4187-8237-38a3091aad08 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.360586 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:06.375108 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d141f-e371-4cf1-bbd8-91cf9b8ffd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.375712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d141f-e371-4cf1-bbd8-91cf9b8ffd8f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.375985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d141f-e371-4cf1-bbd8-91cf9b8ffd8f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.376252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d141f-e371-4cf1-bbd8-91cf9b8ffd8f None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.376886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d141f-e371-4cf1-bbd8-91cf9b8ffd8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:06.377215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d141f-e371-4cf1-bbd8-91cf9b8ffd8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:06.378579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720d141f-e371-4cf1-bbd8-91cf9b8ffd8f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:06.391040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 291/576] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:06.391837 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d141f-e371-4cf1-bbd8-91cf9b8ffd8f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.423549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:06.430385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09794f09-c67e-4dde-9dc6-b701e3e33585 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.465326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def673d9-6395-4501-89bf-278291739467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.466108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def673d9-6395-4501-89bf-278291739467 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.466410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def673d9-6395-4501-89bf-278291739467 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.466681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def673d9-6395-4501-89bf-278291739467 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.467324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def673d9-6395-4501-89bf-278291739467 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:06.467630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def673d9-6395-4501-89bf-278291739467 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:06.469003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def673d9-6395-4501-89bf-278291739467 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:06.480697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 286/577] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:06.481644 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def673d9-6395-4501-89bf-278291739467 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.493517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d141f-e371-4cf1-bbd8-91cf9b8ffd8f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.527556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fc80c-f5fa-4890-afe6-aeafeb3f0293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.528171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fc80c-f5fa-4890-afe6-aeafeb3f0293 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.528447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fc80c-f5fa-4890-afe6-aeafeb3f0293 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.528715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3fc80c-f5fa-4890-afe6-aeafeb3f0293 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.538871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3fc80c-f5fa-4890-afe6-aeafeb3f0293 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-0b3fc80c-f5fa-4890-afe6-aeafeb3f0293 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:06.543602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:06.544585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 292/578] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:06.545471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fc80c-f5fa-4890-afe6-aeafeb3f0293 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.583194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def673d9-6395-4501-89bf-278291739467 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.587452 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:06.618441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359de2f-c2bd-4ea0-ada7-5c271aef14a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.619294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359de2f-c2bd-4ea0-ada7-5c271aef14a3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.619565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359de2f-c2bd-4ea0-ada7-5c271aef14a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.620348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b359de2f-c2bd-4ea0-ada7-5c271aef14a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.621176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359de2f-c2bd-4ea0-ada7-5c271aef14a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:06.621482 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359de2f-c2bd-4ea0-ada7-5c271aef14a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:06.622875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b359de2f-c2bd-4ea0-ada7-5c271aef14a3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:06.633620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 287/579] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:06.634495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359de2f-c2bd-4ea0-ada7-5c271aef14a3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.646186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3fc80c-f5fa-4890-afe6-aeafeb3f0293 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.651249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:06.678879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce73b65-74ad-4e85-b1a7-08fac356491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.679474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce73b65-74ad-4e85-b1a7-08fac356491a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.679750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce73b65-74ad-4e85-b1a7-08fac356491a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.680020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce73b65-74ad-4e85-b1a7-08fac356491a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.680732 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce73b65-74ad-4e85-b1a7-08fac356491a None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=471e7550ae7a4f909851215d3ed148c6) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:06.681037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce73b65-74ad-4e85-b1a7-08fac356491a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:06.682985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce73b65-74ad-4e85-b1a7-08fac356491a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:06.688314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 293/580] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/role_assignments?user.id=471e7550ae7a4f909851215d3ed148c6&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:06.689119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce73b65-74ad-4e85-b1a7-08fac356491a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.741553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b359de2f-c2bd-4ea0-ada7-5c271aef14a3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.777444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178e18f-964f-458e-82df-1805d01ed003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.778085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e18f-964f-458e-82df-1805d01ed003 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.778363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e18f-964f-458e-82df-1805d01ed003 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.778633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4178e18f-964f-458e-82df-1805d01ed003 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.785967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178e18f-964f-458e-82df-1805d01ed003 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:06.787432 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4178e18f-964f-458e-82df-1805d01ed003 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-4178e18f-964f-458e-82df-1805d01ed003 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:06.799827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:06.800799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:06.801797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:06.801797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:06.801797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:06.801797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:06.801797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:06.801797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:06.801797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:06.801797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:06.801797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 288/581] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:06.801797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178e18f-964f-458e-82df-1805d01ed003 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.814304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce73b65-74ad-4e85-b1a7-08fac356491a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.817515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:06.854949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78758a24-0c13-462e-98b3-957641fefa01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.855614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78758a24-0c13-462e-98b3-957641fefa01 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.855865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78758a24-0c13-462e-98b3-957641fefa01 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.856128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78758a24-0c13-462e-98b3-957641fefa01 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.863933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78758a24-0c13-462e-98b3-957641fefa01 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:06.865399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78758a24-0c13-462e-98b3-957641fefa01 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-78758a24-0c13-462e-98b3-957641fefa01 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:06.878988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:06.880162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 294/582] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:06.881490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78758a24-0c13-462e-98b3-957641fefa01 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.900466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:06.928760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4178e18f-964f-458e-82df-1805d01ed003 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:06.963222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5da664-fbc5-4fee-81a1-634936502b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:06.963853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5da664-fbc5-4fee-81a1-634936502b48 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:06.964130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5da664-fbc5-4fee-81a1-634936502b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:06.964401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5da664-fbc5-4fee-81a1-634936502b48 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:06.965058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5da664-fbc5-4fee-81a1-634936502b48 None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:06.965392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5da664-fbc5-4fee-81a1-634936502b48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:06.967004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5da664-fbc5-4fee-81a1-634936502b48 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:06.975155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 289/583] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:06.975941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5da664-fbc5-4fee-81a1-634936502b48 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:06.997385 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78758a24-0c13-462e-98b3-957641fefa01 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:07.034561 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d02ab7-74e2-46b5-86f6-d7636805c300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:07.035185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d02ab7-74e2-46b5-86f6-d7636805c300 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:07.035458 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d02ab7-74e2-46b5-86f6-d7636805c300 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:07.035725 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d02ab7-74e2-46b5-86f6-d7636805c300 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:07.044268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d02ab7-74e2-46b5-86f6-d7636805c300 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:07.045737 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d02ab7-74e2-46b5-86f6-d7636805c300 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-00d02ab7-74e2-46b5-86f6-d7636805c300 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:07.059962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:07.060872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:07.061842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:07.061842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:07.061842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:07.061842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:07.061842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:07.061842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:07.061842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:07.061842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:07.061842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 21:57:07.061842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:07.061842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 295/584] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:07.062430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d02ab7-74e2-46b5-86f6-d7636805c300 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:07.088007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5da664-fbc5-4fee-81a1-634936502b48 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:07.092846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:07.124993 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfff68-5a0b-44be-a6ab-80792f355b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:07.125789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfff68-5a0b-44be-a6ab-80792f355b9e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:07.126066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfff68-5a0b-44be-a6ab-80792f355b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:07.126336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bfff68-5a0b-44be-a6ab-80792f355b9e None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:07.127023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfff68-5a0b-44be-a6ab-80792f355b9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:07.127316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfff68-5a0b-44be-a6ab-80792f355b9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:07.128738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bfff68-5a0b-44be-a6ab-80792f355b9e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:07.131472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:07.144136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 290/585] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:07.145229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfff68-5a0b-44be-a6ab-80792f355b9e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:07.179376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d02ab7-74e2-46b5-86f6-d7636805c300 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:07.217148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c490890-b26b-4a33-92e3-26d55b981cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:07.217148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c490890-b26b-4a33-92e3-26d55b981cab None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:07.217148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c490890-b26b-4a33-92e3-26d55b981cab None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:07.217148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c490890-b26b-4a33-92e3-26d55b981cab None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:07.217580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c490890-b26b-4a33-92e3-26d55b981cab None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:07.217580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c490890-b26b-4a33-92e3-26d55b981cab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:07.218405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c490890-b26b-4a33-92e3-26d55b981cab None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:07.241609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 296/586] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 22 21:57:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:07.242508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c490890-b26b-4a33-92e3-26d55b981cab None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:07.274845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bfff68-5a0b-44be-a6ab-80792f355b9e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:07.310262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5900fc8-4401-483b-be82-32f6c7b62178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:07.310877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5900fc8-4401-483b-be82-32f6c7b62178 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:07.311157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5900fc8-4401-483b-be82-32f6c7b62178 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:07.311429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5900fc8-4401-483b-be82-32f6c7b62178 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:07.322983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5900fc8-4401-483b-be82-32f6c7b62178 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:07.324437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5900fc8-4401-483b-be82-32f6c7b62178 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:07.331328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a5900fc8-4401-483b-be82-32f6c7b62178 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:07.340484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:07.341750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:07.342654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:07.342654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:07.342654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:07.342654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 291/587] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:07.342654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5900fc8-4401-483b-be82-32f6c7b62178 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:07.358206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c490890-b26b-4a33-92e3-26d55b981cab None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:07.391788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:07.395298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add2b9c-311c-4070-a9c4-11409910e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:07.396422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add2b9c-311c-4070-a9c4-11409910e345 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:07.396871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add2b9c-311c-4070-a9c4-11409910e345 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:07.397167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add2b9c-311c-4070-a9c4-11409910e345 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:07.397897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add2b9c-311c-4070-a9c4-11409910e345 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:07.398200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add2b9c-311c-4070-a9c4-11409910e345 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:07.399869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9add2b9c-311c-4070-a9c4-11409910e345 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:07.424406 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 297/588] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 22 21:57:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:07.424406 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add2b9c-311c-4070-a9c4-11409910e345 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:07.448564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5900fc8-4401-483b-be82-32f6c7b62178 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:07.483000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678ec21-2943-482c-97a2-129179c6f8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:07.483746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678ec21-2943-482c-97a2-129179c6f8c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:07.484024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678ec21-2943-482c-97a2-129179c6f8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:07.484298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5678ec21-2943-482c-97a2-129179c6f8c4 None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/195565a7695b4d9b82e0ee76181fee2f/roles/054c6569cc994d1c950d5e554273033e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:07.499528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678ec21-2943-482c-97a2-129179c6f8c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=195565a7695b4d9b82e0ee76181fee2f, role_id=054c6569cc994d1c950d5e554273033e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:07.500959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5678ec21-2943-482c-97a2-129179c6f8c4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:07.521146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 292/589] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:57:07 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/195565a7695b4d9b82e0ee76181fee2f/roles/054c6569cc994d1c950d5e554273033e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:57:07.521146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678ec21-2943-482c-97a2-129179c6f8c4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:07.527736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add2b9c-311c-4070-a9c4-11409910e345 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:07.571455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e48ab5-219c-4db8-b7de-9ccfebf5ad66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:07.572310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48ab5-219c-4db8-b7de-9ccfebf5ad66 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:07.572502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48ab5-219c-4db8-b7de-9ccfebf5ad66 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:07.572917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e48ab5-219c-4db8-b7de-9ccfebf5ad66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:07.584095 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:07.586914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e48ab5-219c-4db8-b7de-9ccfebf5ad66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:07.589034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e48ab5-219c-4db8-b7de-9ccfebf5ad66 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b6e48ab5-219c-4db8-b7de-9ccfebf5ad66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:07.609079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:07.610128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:07.611028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:07.611028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:07.611028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:07.611028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 298/590] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:07.611251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e48ab5-219c-4db8-b7de-9ccfebf5ad66 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:07.675315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:07.682017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5678ec21-2943-482c-97a2-129179c6f8c4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:07.736000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e48ab5-219c-4db8-b7de-9ccfebf5ad66 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:07.738864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2135d70-57b0-4ce1-bfbe-adf9885204e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:07.740316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2135d70-57b0-4ce1-bfbe-adf9885204e1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:07.740590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2135d70-57b0-4ce1-bfbe-adf9885204e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:07.740859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2135d70-57b0-4ce1-bfbe-adf9885204e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:07.742205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2135d70-57b0-4ce1-bfbe-adf9885204e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:07.742205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2135d70-57b0-4ce1-bfbe-adf9885204e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:07.743190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2135d70-57b0-4ce1-bfbe-adf9885204e1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:07.761969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 293/591] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:07.766096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2135d70-57b0-4ce1-bfbe-adf9885204e1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:07.781361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434993-2d9e-4efd-b721-ae2f300849da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:07.782365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434993-2d9e-4efd-b721-ae2f300849da None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:07.782881 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434993-2d9e-4efd-b721-ae2f300849da None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:07.783351 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434993-2d9e-4efd-b721-ae2f300849da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:07.797188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33434993-2d9e-4efd-b721-ae2f300849da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:07.799497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33434993-2d9e-4efd-b721-ae2f300849da None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-33434993-2d9e-4efd-b721-ae2f300849da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:07.815961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:07.817162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:07.818254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:07.818254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:07.818254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:07.819921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 299/592] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:07.825256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434993-2d9e-4efd-b721-ae2f300849da None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:07.879447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:07.899479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2135d70-57b0-4ce1-bfbe-adf9885204e1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:07.946504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434993-2d9e-4efd-b721-ae2f300849da None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:07.948112 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c45ee9-09f7-407b-b8f1-8011b0564aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:07.949381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c45ee9-09f7-407b-b8f1-8011b0564aac None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:07.949794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c45ee9-09f7-407b-b8f1-8011b0564aac None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:07.951101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c45ee9-09f7-407b-b8f1-8011b0564aac None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:07.951858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c45ee9-09f7-407b-b8f1-8011b0564aac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:07.952264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c45ee9-09f7-407b-b8f1-8011b0564aac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:07.953793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c45ee9-09f7-407b-b8f1-8011b0564aac None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:07.970700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 294/593] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:07.971969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c45ee9-09f7-407b-b8f1-8011b0564aac None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:07.986111 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2f0ba-af3e-4c85-bdec-880736f46c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:07.986749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2f0ba-af3e-4c85-bdec-880736f46c0a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:07.987035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2f0ba-af3e-4c85-bdec-880736f46c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:07.987417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e2f0ba-af3e-4c85-bdec-880736f46c0a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:07.996259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2f0ba-af3e-4c85-bdec-880736f46c0a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:07.997539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:07.997779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e2f0ba-af3e-4c85-bdec-880736f46c0a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-97e2f0ba-af3e-4c85-bdec-880736f46c0a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:08.010212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:08.011426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:08.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:08.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:08.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:08.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:08.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:08.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:08.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:08.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:08.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 21:57:08.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:08.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 300/594] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:08.012940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2f0ba-af3e-4c85-bdec-880736f46c0a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:08.124435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c45ee9-09f7-407b-b8f1-8011b0564aac None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:08.145128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e2f0ba-af3e-4c85-bdec-880736f46c0a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:08.148226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:08.175157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6091b9-0c86-4f03-886d-204c0c37989a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:08.175751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6091b9-0c86-4f03-886d-204c0c37989a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:08.176028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6091b9-0c86-4f03-886d-204c0c37989a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:08.176305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6091b9-0c86-4f03-886d-204c0c37989a None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:08.176929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6091b9-0c86-4f03-886d-204c0c37989a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:08.177256 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6091b9-0c86-4f03-886d-204c0c37989a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:08.179300 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6091b9-0c86-4f03-886d-204c0c37989a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:08.195625 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 295/595] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:08.195625 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6091b9-0c86-4f03-886d-204c0c37989a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:08.198853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3575b-10de-4d72-9410-92e5d90e2421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:08.199718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3575b-10de-4d72-9410-92e5d90e2421 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:08.200166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3575b-10de-4d72-9410-92e5d90e2421 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:08.200583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e3575b-10de-4d72-9410-92e5d90e2421 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:08.210051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3575b-10de-4d72-9410-92e5d90e2421 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:08.212438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e3575b-10de-4d72-9410-92e5d90e2421 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-09e3575b-10de-4d72-9410-92e5d90e2421 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:08.229987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:08.230993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:08.231902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:08.231902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:08.231902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:08.231902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:08.231902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:08.231902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:08.231902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:08.231902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:08.234938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 301/596] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:08.235869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3575b-10de-4d72-9410-92e5d90e2421 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:08.295918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:08.330056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6091b9-0c86-4f03-886d-204c0c37989a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:08.349022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e3575b-10de-4d72-9410-92e5d90e2421 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:08.377388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78b11e-2c08-4f92-b664-6b8dd046d3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:08.378635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78b11e-2c08-4f92-b664-6b8dd046d3be None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:08.379092 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78b11e-2c08-4f92-b664-6b8dd046d3be None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:08.379513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78b11e-2c08-4f92-b664-6b8dd046d3be None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:08.380386 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78b11e-2c08-4f92-b664-6b8dd046d3be None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:08.380848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78b11e-2c08-4f92-b664-6b8dd046d3be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:08.383004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78b11e-2c08-4f92-b664-6b8dd046d3be None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:08.398293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e9860-e08b-4a50-af16-fa8bf1830171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:08.399106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e9860-e08b-4a50-af16-fa8bf1830171 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:08.399547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e9860-e08b-4a50-af16-fa8bf1830171 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:08.399931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451e9860-e08b-4a50-af16-fa8bf1830171 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:08.406351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 296/597] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 22 21:57:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:08.407708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78b11e-2c08-4f92-b664-6b8dd046d3be None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:08.407917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e9860-e08b-4a50-af16-fa8bf1830171 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:08.409745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451e9860-e08b-4a50-af16-fa8bf1830171 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-451e9860-e08b-4a50-af16-fa8bf1830171 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:08.423781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:08.425006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:08.426155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:08.426155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:08.426155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:08.426155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:08.426155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:08.426155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:08.426155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:08.426155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:08.432921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 302/598] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:08.432921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e9860-e08b-4a50-af16-fa8bf1830171 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:08.466137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:08.524602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78b11e-2c08-4f92-b664-6b8dd046d3be None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:08.561344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d7c18-0502-4d5b-8bac-36450904a4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:08.562344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d7c18-0502-4d5b-8bac-36450904a4e4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:08.562344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d7c18-0502-4d5b-8bac-36450904a4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:08.562774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805d7c18-0502-4d5b-8bac-36450904a4e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:08.563619 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d7c18-0502-4d5b-8bac-36450904a4e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:08.563938 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d7c18-0502-4d5b-8bac-36450904a4e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:08.565468 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805d7c18-0502-4d5b-8bac-36450904a4e4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:08.568049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:08.569612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451e9860-e08b-4a50-af16-fa8bf1830171 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:08.580025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 297/599] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:08.581070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d7c18-0502-4d5b-8bac-36450904a4e4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:08.611697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a8877-4b4a-4acd-a82d-6bd723c6d13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:08.612870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a8877-4b4a-4acd-a82d-6bd723c6d13c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:08.613599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a8877-4b4a-4acd-a82d-6bd723c6d13c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:08.614045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a8877-4b4a-4acd-a82d-6bd723c6d13c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:08.629631 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a8877-4b4a-4acd-a82d-6bd723c6d13c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:08.631765 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869a8877-4b4a-4acd-a82d-6bd723c6d13c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-869a8877-4b4a-4acd-a82d-6bd723c6d13c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:08.652856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:08.654056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:08.655152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:08.655152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:08.655152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:08.657420 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 303/600] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:08.658747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a8877-4b4a-4acd-a82d-6bd723c6d13c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:08.707743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805d7c18-0502-4d5b-8bac-36450904a4e4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:08.749314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a55a7-c05b-4121-8070-d78b5eaf16df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:08.750178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a55a7-c05b-4121-8070-d78b5eaf16df None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:08.751011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a55a7-c05b-4121-8070-d78b5eaf16df None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:08.751400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a55a7-c05b-4121-8070-d78b5eaf16df None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:08.752453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a55a7-c05b-4121-8070-d78b5eaf16df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:08.752854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a55a7-c05b-4121-8070-d78b5eaf16df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:08.755396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a55a7-c05b-4121-8070-d78b5eaf16df None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:08.763203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:08.775307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 298/601] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:08.775307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a55a7-c05b-4121-8070-d78b5eaf16df None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:08.800086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a8877-4b4a-4acd-a82d-6bd723c6d13c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:08.847935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b80038-1e20-478a-9fbf-14dd007a9fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:08.848621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:08.848989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b80038-1e20-478a-9fbf-14dd007a9fdd None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:08.849440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b80038-1e20-478a-9fbf-14dd007a9fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:08.850289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b80038-1e20-478a-9fbf-14dd007a9fdd None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/d011e54e12674eaeb82c68da628ecb9d/roles/054c6569cc994d1c950d5e554273033e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:08.867934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b80038-1e20-478a-9fbf-14dd007a9fdd None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=d011e54e12674eaeb82c68da628ecb9d, role_id=054c6569cc994d1c950d5e554273033e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:08.870305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b80038-1e20-478a-9fbf-14dd007a9fdd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:08.887546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 304/602] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:57:08 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/d011e54e12674eaeb82c68da628ecb9d/roles/054c6569cc994d1c950d5e554273033e => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:57:08.889959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b80038-1e20-478a-9fbf-14dd007a9fdd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:08.902613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a55a7-c05b-4121-8070-d78b5eaf16df None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:08.949600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95d560-a7fd-4b26-affc-ed61dd019f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:08.950221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95d560-a7fd-4b26-affc-ed61dd019f1c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:08.950504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95d560-a7fd-4b26-affc-ed61dd019f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:08.950775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95d560-a7fd-4b26-affc-ed61dd019f1c None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:08.951423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95d560-a7fd-4b26-affc-ed61dd019f1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:08.951727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95d560-a7fd-4b26-affc-ed61dd019f1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:08.953609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95d560-a7fd-4b26-affc-ed61dd019f1c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:08.973322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 299/603] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:08.974944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d93ea-af98-412b-a8c2-a9800ff044d7 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:08.976243 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d93ea-af98-412b-a8c2-a9800ff044d7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:08.977306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204d93ea-af98-412b-a8c2-a9800ff044d7 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:08.979368 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 300/604] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:08.983749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d93ea-af98-412b-a8c2-a9800ff044d7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:09.037764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b80038-1e20-478a-9fbf-14dd007a9fdd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:09.077578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:09.091667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b27a13-ebe8-416f-b5e8-ebe65db0601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:09.092585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b27a13-ebe8-416f-b5e8-ebe65db0601f None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:09.093007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b27a13-ebe8-416f-b5e8-ebe65db0601f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:09.093464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b27a13-ebe8-416f-b5e8-ebe65db0601f None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/fcfdbb6b3dd04478ae99681cc649fdb9/roles/054c6569cc994d1c950d5e554273033e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:09.116805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b27a13-ebe8-416f-b5e8-ebe65db0601f None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=fcfdbb6b3dd04478ae99681cc649fdb9, role_id=054c6569cc994d1c950d5e554273033e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:09.118383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b27a13-ebe8-416f-b5e8-ebe65db0601f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:09.132875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204d93ea-af98-412b-a8c2-a9800ff044d7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:09.133867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 305/605] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:57:08 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/fcfdbb6b3dd04478ae99681cc649fdb9/roles/054c6569cc994d1c950d5e554273033e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:57:09.136245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8bf7f-7313-431a-b78b-cfca73984248 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:09.136574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8bf7f-7313-431a-b78b-cfca73984248 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:09.136891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8bf7f-7313-431a-b78b-cfca73984248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:09.157249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5ca8bf7f-7313-431a-b78b-cfca73984248 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:09.158589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5ca8bf7f-7313-431a-b78b-cfca73984248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:09.169812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:09.208268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffac02-0ef1-4efc-80a3-7b6fa932f405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:09.209690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffac02-0ef1-4efc-80a3-7b6fa932f405 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:09.210843 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffac02-0ef1-4efc-80a3-7b6fa932f405 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:09.211504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffac02-0ef1-4efc-80a3-7b6fa932f405 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:09.227648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffac02-0ef1-4efc-80a3-7b6fa932f405 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:09.232485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffac02-0ef1-4efc-80a3-7b6fa932f405 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-c6ffac02-0ef1-4efc-80a3-7b6fa932f405 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:09.260923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:09.262331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:09.263424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:09.263424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:09.263424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:09.263424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:09.263424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:09.263424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:09.263424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:09.263424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:09.266936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 301/606] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:09.272565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffac02-0ef1-4efc-80a3-7b6fa932f405 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:09.297807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 306/607] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:09.413063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffac02-0ef1-4efc-80a3-7b6fa932f405 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:09.452494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a10be-e29d-4d3b-a7bd-7122e854510c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:09.453393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a10be-e29d-4d3b-a7bd-7122e854510c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:09.453808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a10be-e29d-4d3b-a7bd-7122e854510c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:09.454233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a10be-e29d-4d3b-a7bd-7122e854510c None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:09.455062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a10be-e29d-4d3b-a7bd-7122e854510c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:09.455503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a10be-e29d-4d3b-a7bd-7122e854510c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:09.457189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a10be-e29d-4d3b-a7bd-7122e854510c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:09.471161 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 302/608] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:09.482391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8bf7f-7313-431a-b78b-cfca73984248 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:09.633063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8bf7f-7313-431a-b78b-cfca73984248 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:09.643219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:09.682411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f033ccf-4afd-4373-8a4b-b726b23bc16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:09.683074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033ccf-4afd-4373-8a4b-b726b23bc16b None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:09.683427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033ccf-4afd-4373-8a4b-b726b23bc16b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:09.683704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f033ccf-4afd-4373-8a4b-b726b23bc16b None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/18c956fa035f40fa88fbf662541121ab/roles/054c6569cc994d1c950d5e554273033e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:09.705260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f033ccf-4afd-4373-8a4b-b726b23bc16b None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=18c956fa035f40fa88fbf662541121ab, role_id=054c6569cc994d1c950d5e554273033e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:09.706778 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f033ccf-4afd-4373-8a4b-b726b23bc16b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:09.724152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 307/609] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:57:09 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/18c956fa035f40fa88fbf662541121ab/roles/054c6569cc994d1c950d5e554273033e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:57:09.803394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a10be-e29d-4d3b-a7bd-7122e854510c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:09.852877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:09.941946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a10be-e29d-4d3b-a7bd-7122e854510c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:09.980698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087c0d9-c79c-4adc-b17a-2dcb346aa312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:09.981366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087c0d9-c79c-4adc-b17a-2dcb346aa312 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:09.981650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087c0d9-c79c-4adc-b17a-2dcb346aa312 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:09.981915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087c0d9-c79c-4adc-b17a-2dcb346aa312 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:09.993717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087c0d9-c79c-4adc-b17a-2dcb346aa312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:09.996610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359068c-89f0-4625-8b63-93312ce55b1c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:09.997055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359068c-89f0-4625-8b63-93312ce55b1c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:09.997198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9359068c-89f0-4625-8b63-93312ce55b1c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:09.998674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 308/610] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f087c0d9-c79c-4adc-b17a-2dcb346aa312 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:09.998898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:09.999803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:10.001072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 303/611] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:10.004761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7f3c7-8962-4ce4-84ba-901d01013c5e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:10.005380 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7f3c7-8962-4ce4-84ba-901d01013c5e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:10.005728 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e7f3c7-8962-4ce4-84ba-901d01013c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:10.006001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087c0d9-c79c-4adc-b17a-2dcb346aa312 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:10.040305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b1e7f3c7-8962-4ce4-84ba-901d01013c5e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:10.043273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b1e7f3c7-8962-4ce4-84ba-901d01013c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:10.157548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:10.166901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087c0d9-c79c-4adc-b17a-2dcb346aa312 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:10.186870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 309/612] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:10.220497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37682e68-9cc1-4728-8cab-1de588b0fcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:10.221160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37682e68-9cc1-4728-8cab-1de588b0fcbc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:10.221362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37682e68-9cc1-4728-8cab-1de588b0fcbc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:10.221613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37682e68-9cc1-4728-8cab-1de588b0fcbc None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:10.222274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37682e68-9cc1-4728-8cab-1de588b0fcbc None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:10.222580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37682e68-9cc1-4728-8cab-1de588b0fcbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:10.223950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37682e68-9cc1-4728-8cab-1de588b0fcbc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:10.233161 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 304/613] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:10.238632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7f3c7-8962-4ce4-84ba-901d01013c5e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:10.329348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:10.346738 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e7f3c7-8962-4ce4-84ba-901d01013c5e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:10.382200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee75cea-ebaa-4043-8dd1-8d221c39bad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:10.382834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee75cea-ebaa-4043-8dd1-8d221c39bad8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:10.383106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee75cea-ebaa-4043-8dd1-8d221c39bad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:10.383372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee75cea-ebaa-4043-8dd1-8d221c39bad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:10.395715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee75cea-ebaa-4043-8dd1-8d221c39bad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:10.397189 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee75cea-ebaa-4043-8dd1-8d221c39bad8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1ee75cea-ebaa-4043-8dd1-8d221c39bad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:10.413091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:10.414134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:10.415442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:10.415442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:10.415442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:10.415442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 310/614] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:10.422272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37682e68-9cc1-4728-8cab-1de588b0fcbc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:10.549717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37682e68-9cc1-4728-8cab-1de588b0fcbc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:10.584002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d0fc1-1268-4133-97d4-fb7f2a9a3409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:10.584609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d0fc1-1268-4133-97d4-fb7f2a9a3409 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:10.584882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d0fc1-1268-4133-97d4-fb7f2a9a3409 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:10.585177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d0fc1-1268-4133-97d4-fb7f2a9a3409 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:10.585807 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d0fc1-1268-4133-97d4-fb7f2a9a3409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:10.586105 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d0fc1-1268-4133-97d4-fb7f2a9a3409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:10.587592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d0fc1-1268-4133-97d4-fb7f2a9a3409 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:10.589010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee75cea-ebaa-4043-8dd1-8d221c39bad8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:10.601909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 305/615] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:10.610563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d0fc1-1268-4133-97d4-fb7f2a9a3409 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:10.647223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:10.802470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee75cea-ebaa-4043-8dd1-8d221c39bad8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:10.819377 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d0fc1-1268-4133-97d4-fb7f2a9a3409 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:10.870779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e6bbd-82a4-49d2-b925-abd06a50eead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:10.873420 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6bbd-82a4-49d2-b925-abd06a50eead None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:10.874899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6bbd-82a4-49d2-b925-abd06a50eead None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:10.876000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2e6bbd-82a4-49d2-b925-abd06a50eead None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:10.884226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2d9c9-d0a7-4a3d-9815-ee755df7ca9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:10.885662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2d9c9-d0a7-4a3d-9815-ee755df7ca9a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:10.886351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2d9c9-d0a7-4a3d-9815-ee755df7ca9a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:10.886768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef2d9c9-d0a7-4a3d-9815-ee755df7ca9a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:10.888908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:10.894835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2e6bbd-82a4-49d2-b925-abd06a50eead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2d2e6bbd-82a4-49d2-b925-abd06a50eead None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:10.903152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:10.904298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:10.905374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:10.907585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2d9c9-d0a7-4a3d-9815-ee755df7ca9a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:10.909212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef2d9c9-d0a7-4a3d-9815-ee755df7ca9a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:10.910368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 311/616] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:10.918072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e6bbd-82a4-49d2-b925-abd06a50eead None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-cef2d9c9-d0a7-4a3d-9815-ee755df7ca9a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:10.926267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:10.927343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:10.928428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:10.928428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:10.928428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:10.928428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:10.928428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:10.928428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:10.928428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:10.928428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:10.928428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:10.928428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:10.937426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 306/617] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:10.941868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2d9c9-d0a7-4a3d-9815-ee755df7ca9a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:11.100042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:11.120427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2e6bbd-82a4-49d2-b925-abd06a50eead None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:11.147637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef2d9c9-d0a7-4a3d-9815-ee755df7ca9a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:11.192721 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3dfd43-2e20-4c31-93a2-1d34384ddc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:11.193641 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dfd43-2e20-4c31-93a2-1d34384ddc81 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:11.194068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dfd43-2e20-4c31-93a2-1d34384ddc81 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:11.194489 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3dfd43-2e20-4c31-93a2-1d34384ddc81 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:11.195311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3dfd43-2e20-4c31-93a2-1d34384ddc81 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:11.195763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3dfd43-2e20-4c31-93a2-1d34384ddc81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:11.198091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3dfd43-2e20-4c31-93a2-1d34384ddc81 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:11.209317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 312/618] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:11.214053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3dfd43-2e20-4c31-93a2-1d34384ddc81 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:11.227030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb59a9-98de-41b2-905e-e5d581bdec10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:11.228217 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb59a9-98de-41b2-905e-e5d581bdec10 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:11.229219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb59a9-98de-41b2-905e-e5d581bdec10 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:11.229762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdb59a9-98de-41b2-905e-e5d581bdec10 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:11.230900 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb59a9-98de-41b2-905e-e5d581bdec10 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:11.233124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb59a9-98de-41b2-905e-e5d581bdec10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:11.234737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdb59a9-98de-41b2-905e-e5d581bdec10 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:11.267624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 307/619] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 22 21:57:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:11.274733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb59a9-98de-41b2-905e-e5d581bdec10 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:11.326273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:11.399972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3dfd43-2e20-4c31-93a2-1d34384ddc81 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:11.465913 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdb59a9-98de-41b2-905e-e5d581bdec10 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:11.469698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6f3a7b-6fe1-4b46-903c-1c6dee3aebdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:11.470574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f3a7b-6fe1-4b46-903c-1c6dee3aebdf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:11.471408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f3a7b-6fe1-4b46-903c-1c6dee3aebdf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:11.472029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6f3a7b-6fe1-4b46-903c-1c6dee3aebdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:11.495027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f3a7b-6fe1-4b46-903c-1c6dee3aebdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:11.497835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6f3a7b-6fe1-4b46-903c-1c6dee3aebdf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5a6f3a7b-6fe1-4b46-903c-1c6dee3aebdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:11.525082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:11.526370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:11.527409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:11.527409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:11.527409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:11.534066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 313/620] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:11.535562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248bf5c-7cdc-428e-ac7a-dafe1b940fd6 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:11.535949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248bf5c-7cdc-428e-ac7a-dafe1b940fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:11.536233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248bf5c-7cdc-428e-ac7a-dafe1b940fd6 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:11.537343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd798097-84a7-49e2-b3bf-3be07a09b723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:11.537734 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 314/621] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:11.538327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd798097-84a7-49e2-b3bf-3be07a09b723 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:11.538768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd798097-84a7-49e2-b3bf-3be07a09b723 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:11.539192 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd798097-84a7-49e2-b3bf-3be07a09b723 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:11.542894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248bf5c-7cdc-428e-ac7a-dafe1b940fd6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:11.543358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:11.552193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd798097-84a7-49e2-b3bf-3be07a09b723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:11.553911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd798097-84a7-49e2-b3bf-3be07a09b723 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-dd798097-84a7-49e2-b3bf-3be07a09b723 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:11.570291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:11.571444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:11.572587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:11.572587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:11.572587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:11.574271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 308/622] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:11.575708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48aede-beea-4ee4-a3e8-9a1ab4cb75b2 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:11.576216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48aede-beea-4ee4-a3e8-9a1ab4cb75b2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:11.576689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48aede-beea-4ee4-a3e8-9a1ab4cb75b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:11.596026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-cd48aede-beea-4ee4-a3e8-9a1ab4cb75b2 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:11.597572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cd48aede-beea-4ee4-a3e8-9a1ab4cb75b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:11.659728 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248bf5c-7cdc-428e-ac7a-dafe1b940fd6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:11.670713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 309/623] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:11.671775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48aede-beea-4ee4-a3e8-9a1ab4cb75b2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:11.688568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:11.732636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ff44a-dfd8-482e-a25b-f1510c886549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:11.733764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ff44a-dfd8-482e-a25b-f1510c886549 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:11.733764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ff44a-dfd8-482e-a25b-f1510c886549 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:11.733974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2ff44a-dfd8-482e-a25b-f1510c886549 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:11.734434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ff44a-dfd8-482e-a25b-f1510c886549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:11.734736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ff44a-dfd8-482e-a25b-f1510c886549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:11.739539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2ff44a-dfd8-482e-a25b-f1510c886549 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:11.765834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 315/624] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:11.771431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ff44a-dfd8-482e-a25b-f1510c886549 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:11.876820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48aede-beea-4ee4-a3e8-9a1ab4cb75b2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:11.919858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:11.934814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2ff44a-dfd8-482e-a25b-f1510c886549 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:11.956809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ce102-2380-45b9-98aa-f8204b27bc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:11.957556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ce102-2380-45b9-98aa-f8204b27bc39 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:11.957833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ce102-2380-45b9-98aa-f8204b27bc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:11.958202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ce102-2380-45b9-98aa-f8204b27bc39 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:11.958842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ce102-2380-45b9-98aa-f8204b27bc39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:11.959144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ce102-2380-45b9-98aa-f8204b27bc39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:11.960620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ce102-2380-45b9-98aa-f8204b27bc39 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:11.976719 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 310/625] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:11.978030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae4886-c908-40ca-be2c-a4900ed3a5bc None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:11.978317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae4886-c908-40ca-be2c-a4900ed3a5bc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:11.979230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddae4886-c908-40ca-be2c-a4900ed3a5bc None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:11.980767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 311/626] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:11.981849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae4886-c908-40ca-be2c-a4900ed3a5bc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:11.982447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a3cbf-34c2-4d48-a0f7-0d4f9e967e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:11.983086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a3cbf-34c2-4d48-a0f7-0d4f9e967e66 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:11.983366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a3cbf-34c2-4d48-a0f7-0d4f9e967e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:11.983638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a3cbf-34c2-4d48-a0f7-0d4f9e967e66 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:11.992634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a3cbf-34c2-4d48-a0f7-0d4f9e967e66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:11.995054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83a3cbf-34c2-4d48-a0f7-0d4f9e967e66 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d83a3cbf-34c2-4d48-a0f7-0d4f9e967e66 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:12.007523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:12.008691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 316/627] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:57:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:12.009753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a3cbf-34c2-4d48-a0f7-0d4f9e967e66 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:12.043243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:12.106871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddae4886-c908-40ca-be2c-a4900ed3a5bc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:12.128191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a3cbf-34c2-4d48-a0f7-0d4f9e967e66 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:12.144158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fb325-e6aa-418a-9c17-a094109950de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:12.144783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fb325-e6aa-418a-9c17-a094109950de None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.145066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fb325-e6aa-418a-9c17-a094109950de None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.145373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fb325-e6aa-418a-9c17-a094109950de None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.155824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fb325-e6aa-418a-9c17-a094109950de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-330fb325-e6aa-418a-9c17-a094109950de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:12.160565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:12.161522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:12.162362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:12.165732 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c94a4-d24c-4230-b65e-87e96380b1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:12.167020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c94a4-d24c-4230-b65e-87e96380b1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.167139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 312/628] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:12.167335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c94a4-d24c-4230-b65e-87e96380b1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.167608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c94a4-d24c-4230-b65e-87e96380b1c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.168579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c675d-bb43-403a-8592-bc4aced17172 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.168866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c675d-bb43-403a-8592-bc4aced17172 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.169168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c675d-bb43-403a-8592-bc4aced17172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.175130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c94a4-d24c-4230-b65e-87e96380b1c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:12.176553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c94a4-d24c-4230-b65e-87e96380b1c3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:12.188201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a76c675d-bb43-403a-8592-bc4aced17172 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:12.189429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a76c675d-bb43-403a-8592-bc4aced17172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2c3c94a4-d24c-4230-b65e-87e96380b1c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:12.189549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:12.190415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:12.196294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:12.196294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:12.196294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:12.196294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:12.196294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:12.196294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:12.196294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:12.196294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:12.196294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 317/629] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:12.196294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c94a4-d24c-4230-b65e-87e96380b1c3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:12.225618 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:12.262903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 313/630] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:12.263549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c675d-bb43-403a-8592-bc4aced17172 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:12.293340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:12.305171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c94a4-d24c-4230-b65e-87e96380b1c3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:12.343150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50e10-4e84-49c6-a503-a31f9d8b2c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:12.343762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50e10-4e84-49c6-a503-a31f9d8b2c44 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.344305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50e10-4e84-49c6-a503-a31f9d8b2c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.344305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca50e10-4e84-49c6-a503-a31f9d8b2c44 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.345011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50e10-4e84-49c6-a503-a31f9d8b2c44 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:12.345343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50e10-4e84-49c6-a503-a31f9d8b2c44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:12.346788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca50e10-4e84-49c6-a503-a31f9d8b2c44 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:12.365853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 318/631] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 22 21:57:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:12.366656 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50e10-4e84-49c6-a503-a31f9d8b2c44 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:12.373582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c675d-bb43-403a-8592-bc4aced17172 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:12.408555 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca1caf-1009-4b82-8d80-bfeaa12984c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:12.409083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca1caf-1009-4b82-8d80-bfeaa12984c0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.409405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca1caf-1009-4b82-8d80-bfeaa12984c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.409674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca1caf-1009-4b82-8d80-bfeaa12984c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.410329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca1caf-1009-4b82-8d80-bfeaa12984c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:12.410631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca1caf-1009-4b82-8d80-bfeaa12984c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:12.411997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fca1caf-1009-4b82-8d80-bfeaa12984c0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:12.424819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 314/632] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:12.426100 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4109af-9091-4f7e-9b19-73bed2149459 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.426389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4109af-9091-4f7e-9b19-73bed2149459 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.426670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4109af-9091-4f7e-9b19-73bed2149459 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.428013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 315/633] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:12.428573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4109af-9091-4f7e-9b19-73bed2149459 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:12.460970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:12.481283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca50e10-4e84-49c6-a503-a31f9d8b2c44 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:12.522192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7ce9a-1412-4232-94a4-558fac8af1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:12.522807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7ce9a-1412-4232-94a4-558fac8af1aa None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.523075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7ce9a-1412-4232-94a4-558fac8af1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.523356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7ce9a-1412-4232-94a4-558fac8af1aa None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.524039 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7ce9a-1412-4232-94a4-558fac8af1aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:12.524345 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7ce9a-1412-4232-94a4-558fac8af1aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:12.526679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7ce9a-1412-4232-94a4-558fac8af1aa None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:12.538239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4109af-9091-4f7e-9b19-73bed2149459 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:12.539922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 319/634] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:12.540596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7ce9a-1412-4232-94a4-558fac8af1aa None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:12.554381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:12.573739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaaeb7-0921-4a80-8a51-e327962fceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:12.574342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaaeb7-0921-4a80-8a51-e327962fceb4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.574617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaaeb7-0921-4a80-8a51-e327962fceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.574887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabaaeb7-0921-4a80-8a51-e327962fceb4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.641546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabaaeb7-0921-4a80-8a51-e327962fceb4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:12.641960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7ce9a-1412-4232-94a4-558fac8af1aa None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:12.643052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabaaeb7-0921-4a80-8a51-e327962fceb4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:12.671642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:12.676466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2765ac0e-a360-4661-bab9-2be9572bbba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:12.677084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765ac0e-a360-4661-bab9-2be9572bbba4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.677386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765ac0e-a360-4661-bab9-2be9572bbba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.677653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2765ac0e-a360-4661-bab9-2be9572bbba4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.687821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765ac0e-a360-4661-bab9-2be9572bbba4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:12.689281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2765ac0e-a360-4661-bab9-2be9572bbba4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2765ac0e-a360-4661-bab9-2be9572bbba4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:12.704802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:12.705764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:12.706647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:12.706647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:12.706647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:12.706647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 320/635] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:12.707360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07eef2-635e-4dcf-8810-4c6a22c00d17 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.707360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07eef2-635e-4dcf-8810-4c6a22c00d17 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.707552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07eef2-635e-4dcf-8810-4c6a22c00d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.726009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5a07eef2-635e-4dcf-8810-4c6a22c00d17 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:12.727186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5a07eef2-635e-4dcf-8810-4c6a22c00d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:12.748080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 316/636] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:57:12.749313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaaeb7-0921-4a80-8a51-e327962fceb4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:12.785688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:12.796832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 321/637] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:12.797661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07eef2-635e-4dcf-8810-4c6a22c00d17 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:12.851375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabaaeb7-0921-4a80-8a51-e327962fceb4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:12.880881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:12.888326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfed9c1-a479-4e53-aceb-516615157f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:12.888947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfed9c1-a479-4e53-aceb-516615157f21 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.889247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfed9c1-a479-4e53-aceb-516615157f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.889519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfed9c1-a479-4e53-aceb-516615157f21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.901052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfed9c1-a479-4e53-aceb-516615157f21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:12.905799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07eef2-635e-4dcf-8810-4c6a22c00d17 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-5dfed9c1-a479-4e53-aceb-516615157f21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:12.906062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:12.906996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:12.907871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 317/638] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:12.908567 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfed9c1-a479-4e53-aceb-516615157f21 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:12.950938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c1c44-f6df-4f81-8c93-5fecb205c445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:12.951520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c1c44-f6df-4f81-8c93-5fecb205c445 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:12.951797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c1c44-f6df-4f81-8c93-5fecb205c445 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:12.952069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0c1c44-f6df-4f81-8c93-5fecb205c445 None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/471e7550ae7a4f909851215d3ed148c6/roles/054c6569cc994d1c950d5e554273033e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:12.967044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c1c44-f6df-4f81-8c93-5fecb205c445 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=471e7550ae7a4f909851215d3ed148c6, role_id=054c6569cc994d1c950d5e554273033e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:12.968727 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0c1c44-f6df-4f81-8c93-5fecb205c445 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:12.982777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 322/639] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:57:12 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/471e7550ae7a4f909851215d3ed148c6/roles/054c6569cc994d1c950d5e554273033e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:57:12.983567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c1c44-f6df-4f81-8c93-5fecb205c445 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:13.044521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfed9c1-a479-4e53-aceb-516615157f21 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:13.047729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:13.107897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb2bb2-6f0a-4c64-99be-0fb3990bcee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:13.108510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2bb2-6f0a-4c64-99be-0fb3990bcee4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:13.108782 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2bb2-6f0a-4c64-99be-0fb3990bcee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:13.109046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb2bb2-6f0a-4c64-99be-0fb3990bcee4 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:13.109718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb2bb2-6f0a-4c64-99be-0fb3990bcee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:13.110015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb2bb2-6f0a-4c64-99be-0fb3990bcee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:13.111411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb2bb2-6f0a-4c64-99be-0fb3990bcee4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:13.124143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 318/640] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:13.124951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb2bb2-6f0a-4c64-99be-0fb3990bcee4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:13.126933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0c1c44-f6df-4f81-8c93-5fecb205c445 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:13.176636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb439b7-cbf9-4d04-ba6e-c2121b9ff9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:13.177289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb439b7-cbf9-4d04-ba6e-c2121b9ff9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:13.177289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb439b7-cbf9-4d04-ba6e-c2121b9ff9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:13.177597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb439b7-cbf9-4d04-ba6e-c2121b9ff9f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:13.186236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:13.192788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb439b7-cbf9-4d04-ba6e-c2121b9ff9f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:13.195696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb439b7-cbf9-4d04-ba6e-c2121b9ff9f2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bcb439b7-cbf9-4d04-ba6e-c2121b9ff9f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:13.213473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:13.214442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:13.216310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:13.216310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:13.216310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:13.216310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 323/641] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:13.216310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb439b7-cbf9-4d04-ba6e-c2121b9ff9f2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:13.244984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb2bb2-6f0a-4c64-99be-0fb3990bcee4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:13.293586 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32706f29-665f-40f9-9b74-02ab6496feed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:13.294890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32706f29-665f-40f9-9b74-02ab6496feed None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:13.295308 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32706f29-665f-40f9-9b74-02ab6496feed None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:13.295732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32706f29-665f-40f9-9b74-02ab6496feed None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:13.296530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32706f29-665f-40f9-9b74-02ab6496feed None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:13.296980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32706f29-665f-40f9-9b74-02ab6496feed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:13.298502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32706f29-665f-40f9-9b74-02ab6496feed None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:13.311824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 319/642] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:13.312874 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32706f29-665f-40f9-9b74-02ab6496feed None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:13.317615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:13.326602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb439b7-cbf9-4d04-ba6e-c2121b9ff9f2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:13.362825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f0f10-0c67-4586-8e9e-6e5ea2492314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:13.363882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f0f10-0c67-4586-8e9e-6e5ea2492314 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:13.364223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f0f10-0c67-4586-8e9e-6e5ea2492314 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:13.364579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888f0f10-0c67-4586-8e9e-6e5ea2492314 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:13.375591 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888f0f10-0c67-4586-8e9e-6e5ea2492314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-888f0f10-0c67-4586-8e9e-6e5ea2492314 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:13.380739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:13.381934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:13.382806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 324/643] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:13.383494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f0f10-0c67-4586-8e9e-6e5ea2492314 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:13.426958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32706f29-665f-40f9-9b74-02ab6496feed None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:13.440693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:13.464675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb22fd-8694-4dfc-a0f6-e89440a8fe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:13.465313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb22fd-8694-4dfc-a0f6-e89440a8fe81 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:13.465957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb22fd-8694-4dfc-a0f6-e89440a8fe81 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:13.466224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfeb22fd-8694-4dfc-a0f6-e89440a8fe81 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:13.474227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb22fd-8694-4dfc-a0f6-e89440a8fe81 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:13.475684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfeb22fd-8694-4dfc-a0f6-e89440a8fe81 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-bfeb22fd-8694-4dfc-a0f6-e89440a8fe81 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:13.488022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:13.488972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 320/644] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 22 21:57:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:13.490073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb22fd-8694-4dfc-a0f6-e89440a8fe81 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:13.494016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888f0f10-0c67-4586-8e9e-6e5ea2492314 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:13.531550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a0277-b2fa-4c8c-b91d-3ecaf557f687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:13.532443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a0277-b2fa-4c8c-b91d-3ecaf557f687 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:13.532913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a0277-b2fa-4c8c-b91d-3ecaf557f687 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:13.533387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a0277-b2fa-4c8c-b91d-3ecaf557f687 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:13.534197 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a0277-b2fa-4c8c-b91d-3ecaf557f687 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:13.534696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a0277-b2fa-4c8c-b91d-3ecaf557f687 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:13.536342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a0277-b2fa-4c8c-b91d-3ecaf557f687 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:13.549715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 325/645] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:13.550906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a0277-b2fa-4c8c-b91d-3ecaf557f687 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:13.566491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:13.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfeb22fd-8694-4dfc-a0f6-e89440a8fe81 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:13.636827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5fe2d-fd69-4e6c-a21a-30b05c215d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:13.637465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5fe2d-fd69-4e6c-a21a-30b05c215d22 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:13.637741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5fe2d-fd69-4e6c-a21a-30b05c215d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:13.638010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c5fe2d-fd69-4e6c-a21a-30b05c215d22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:13.648330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5fe2d-fd69-4e6c-a21a-30b05c215d22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:13.649778 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c5fe2d-fd69-4e6c-a21a-30b05c215d22 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:13.657126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a0277-b2fa-4c8c-b91d-3ecaf557f687 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a0c5fe2d-fd69-4e6c-a21a-30b05c215d22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:13.663727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:13.664651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:13.665543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:13.665543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:13.665543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:13.665543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 321/646] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:13.665780 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5fe2d-fd69-4e6c-a21a-30b05c215d22 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:13.694872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:13.702044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b69e4-8ce5-4d4f-935d-3345afd681bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:13.702676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b69e4-8ce5-4d4f-935d-3345afd681bd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:13.702993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b69e4-8ce5-4d4f-935d-3345afd681bd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:13.703197 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b69e4-8ce5-4d4f-935d-3345afd681bd None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:13.703853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b69e4-8ce5-4d4f-935d-3345afd681bd None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:13.704156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b69e4-8ce5-4d4f-935d-3345afd681bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:13.705552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b69e4-8ce5-4d4f-935d-3345afd681bd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:13.721281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 326/647] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:13.721281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b69e4-8ce5-4d4f-935d-3345afd681bd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:13.782987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c5fe2d-fd69-4e6c-a21a-30b05c215d22 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:13.800598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:13.819850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682fbbd-277b-4c79-93af-2ef04b03aecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:13.820611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682fbbd-277b-4c79-93af-2ef04b03aecc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:13.821224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682fbbd-277b-4c79-93af-2ef04b03aecc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:13.821513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1682fbbd-277b-4c79-93af-2ef04b03aecc None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:13.822426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1682fbbd-277b-4c79-93af-2ef04b03aecc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:13.822761 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1682fbbd-277b-4c79-93af-2ef04b03aecc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:13.824238 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1682fbbd-277b-4c79-93af-2ef04b03aecc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:13.827904 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b69e4-8ce5-4d4f-935d-3345afd681bd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:13.846465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 322/648] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 22 21:57:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:13.846465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682fbbd-277b-4c79-93af-2ef04b03aecc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:13.868779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7318ff40-601e-4c90-97b0-92959d40544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:13.870188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318ff40-601e-4c90-97b0-92959d40544c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:13.870467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318ff40-601e-4c90-97b0-92959d40544c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:13.870749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7318ff40-601e-4c90-97b0-92959d40544c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:13.877871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318ff40-601e-4c90-97b0-92959d40544c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:13.879366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7318ff40-601e-4c90-97b0-92959d40544c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-7318ff40-601e-4c90-97b0-92959d40544c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:13.892834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:13.893933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:13.894817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:13.894817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:13.894817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:13.894817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:13.894817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:13.894817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:13.894817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:13.894817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:13.898797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 327/649] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:13.899488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7318ff40-601e-4c90-97b0-92959d40544c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:13.933166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:13.951967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1682fbbd-277b-4c79-93af-2ef04b03aecc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:13.987754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeacde2-2771-4d2d-8fa3-84b3c5206b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:13.988333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeacde2-2771-4d2d-8fa3-84b3c5206b86 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:13.988609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeacde2-2771-4d2d-8fa3-84b3c5206b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:13.988877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeacde2-2771-4d2d-8fa3-84b3c5206b86 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:13.989525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeacde2-2771-4d2d-8fa3-84b3c5206b86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:13.989821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeacde2-2771-4d2d-8fa3-84b3c5206b86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:13.991172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeacde2-2771-4d2d-8fa3-84b3c5206b86 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:14.002647 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 323/650] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:14.003259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeacde2-2771-4d2d-8fa3-84b3c5206b86 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:14.008428 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7318ff40-601e-4c90-97b0-92959d40544c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:14.044657 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7629aba-152f-48cc-ace1-f2182e701586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:14.045307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7629aba-152f-48cc-ace1-f2182e701586 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:14.045583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7629aba-152f-48cc-ace1-f2182e701586 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:14.045854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7629aba-152f-48cc-ace1-f2182e701586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:14.048501 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:14.057408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7629aba-152f-48cc-ace1-f2182e701586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:14.058682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7629aba-152f-48cc-ace1-f2182e701586 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a7629aba-152f-48cc-ace1-f2182e701586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:14.074863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:14.075876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:14.076763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:14.076763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:14.076763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:14.076763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 328/651] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:14.077394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7629aba-152f-48cc-ace1-f2182e701586 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:14.116819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeacde2-2771-4d2d-8fa3-84b3c5206b86 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:14.152395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc77f7-53e7-4f35-8e13-5f73be160335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:14.153001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc77f7-53e7-4f35-8e13-5f73be160335 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:14.153321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc77f7-53e7-4f35-8e13-5f73be160335 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:14.153586 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dc77f7-53e7-4f35-8e13-5f73be160335 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:14.165770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:14.183917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7629aba-152f-48cc-ace1-f2182e701586 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:14.218689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ccd21-f0dd-40df-8291-1f80cbe15112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:14.219532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ccd21-f0dd-40df-8291-1f80cbe15112 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:14.219809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ccd21-f0dd-40df-8291-1f80cbe15112 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:14.220086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951ccd21-f0dd-40df-8291-1f80cbe15112 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:14.220773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ccd21-f0dd-40df-8291-1f80cbe15112 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:14.221074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ccd21-f0dd-40df-8291-1f80cbe15112 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:14.222496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951ccd21-f0dd-40df-8291-1f80cbe15112 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:14.226355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc77f7-53e7-4f35-8e13-5f73be160335 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:14.227785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dc77f7-53e7-4f35-8e13-5f73be160335 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:14.235668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 329/652] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:14.236768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ccd21-f0dd-40df-8291-1f80cbe15112 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:14.293248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:14.339442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 324/653] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:57:14.341284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc77f7-53e7-4f35-8e13-5f73be160335 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:14.352233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951ccd21-f0dd-40df-8291-1f80cbe15112 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:14.383212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:14.392440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d2dc0-4926-496d-89a7-83715bcbad1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:14.393139 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d2dc0-4926-496d-89a7-83715bcbad1d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:14.393415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d2dc0-4926-496d-89a7-83715bcbad1d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:14.393840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d2dc0-4926-496d-89a7-83715bcbad1d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:14.402521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d2dc0-4926-496d-89a7-83715bcbad1d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:14.404763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d2dc0-4926-496d-89a7-83715bcbad1d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c27d2dc0-4926-496d-89a7-83715bcbad1d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:14.417467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:14.418431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 330/654] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:57:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:14.424701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d2dc0-4926-496d-89a7-83715bcbad1d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:14.441684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dc77f7-53e7-4f35-8e13-5f73be160335 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:14.476339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4c7d8-85c1-4bae-855c-bfd52d65af31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:14.476936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4c7d8-85c1-4bae-855c-bfd52d65af31 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:14.477247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4c7d8-85c1-4bae-855c-bfd52d65af31 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:14.477519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4c7d8-85c1-4bae-855c-bfd52d65af31 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:14.478149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4c7d8-85c1-4bae-855c-bfd52d65af31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:14.478449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4c7d8-85c1-4bae-855c-bfd52d65af31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:14.480331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4c7d8-85c1-4bae-855c-bfd52d65af31 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:14.491102 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 325/655] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:14.491886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4c7d8-85c1-4bae-855c-bfd52d65af31 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:14.536600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d2dc0-4926-496d-89a7-83715bcbad1d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:14.547801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:14.576209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b073ca-a282-4c40-8c8e-bc04bad3ad38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:14.576833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b073ca-a282-4c40-8c8e-bc04bad3ad38 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:14.577137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b073ca-a282-4c40-8c8e-bc04bad3ad38 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:14.577526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b073ca-a282-4c40-8c8e-bc04bad3ad38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:14.578247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b073ca-a282-4c40-8c8e-bc04bad3ad38 None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=195565a7695b4d9b82e0ee76181fee2f) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:14.578555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b073ca-a282-4c40-8c8e-bc04bad3ad38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:14.579969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b073ca-a282-4c40-8c8e-bc04bad3ad38 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:14.586752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 331/656] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:14 2022] GET /identity/v3/role_assignments?user.id=195565a7695b4d9b82e0ee76181fee2f&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:14.587530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b073ca-a282-4c40-8c8e-bc04bad3ad38 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:14.605641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4c7d8-85c1-4bae-855c-bfd52d65af31 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:14.609083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:14.660178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95547e-df55-40ee-babf-fe754dc1ba4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:14.661442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95547e-df55-40ee-babf-fe754dc1ba4c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:14.661928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95547e-df55-40ee-babf-fe754dc1ba4c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:14.662635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95547e-df55-40ee-babf-fe754dc1ba4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:14.677841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95547e-df55-40ee-babf-fe754dc1ba4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:14.679025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95547e-df55-40ee-babf-fe754dc1ba4c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-4e95547e-df55-40ee-babf-fe754dc1ba4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:14.694970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:14.695986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:14.696895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:14.696895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:14.696895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:14.700283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 326/657] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:14.702319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95547e-df55-40ee-babf-fe754dc1ba4c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:14.723696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b073ca-a282-4c40-8c8e-bc04bad3ad38 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:14.764171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea635c1a-7de6-42ea-9e08-b7117d045d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:14.764877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea635c1a-7de6-42ea-9e08-b7117d045d65 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:14.765237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea635c1a-7de6-42ea-9e08-b7117d045d65 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:14.765572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea635c1a-7de6-42ea-9e08-b7117d045d65 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:14.766362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea635c1a-7de6-42ea-9e08-b7117d045d65 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:14.766728 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea635c1a-7de6-42ea-9e08-b7117d045d65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:14.769177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea635c1a-7de6-42ea-9e08-b7117d045d65 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:14.791469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 332/658] 10.222.0.25 () {56 vars in 1137 bytes} [Thu Sep 22 21:57:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:14.792516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea635c1a-7de6-42ea-9e08-b7117d045d65 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:14.831832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:14.840189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95547e-df55-40ee-babf-fe754dc1ba4c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:14.881845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3c90f-dd2e-4058-b302-ca3ad4d19b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:14.882565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3c90f-dd2e-4058-b302-ca3ad4d19b3e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:14.882909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3c90f-dd2e-4058-b302-ca3ad4d19b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:14.883246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3c90f-dd2e-4058-b302-ca3ad4d19b3e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:14.893944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3c90f-dd2e-4058-b302-ca3ad4d19b3e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:14.896241 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3c90f-dd2e-4058-b302-ca3ad4d19b3e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-95d3c90f-dd2e-4058-b302-ca3ad4d19b3e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:14.908902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:14.910138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:14.911303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:14.911303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:14.911303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:14.911303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:14.911303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:14.911303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:14.911303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:14.911303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:14.911303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 22 21:57:14.911303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:14.912377 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 327/659] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:14.917206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3c90f-dd2e-4058-b302-ca3ad4d19b3e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:14.927801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea635c1a-7de6-42ea-9e08-b7117d045d65 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:14.933469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:14.969271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afdfa6-5598-4c99-802e-408b17cfca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:14.970471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afdfa6-5598-4c99-802e-408b17cfca99 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:14.971190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afdfa6-5598-4c99-802e-408b17cfca99 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:14.971629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2afdfa6-5598-4c99-802e-408b17cfca99 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:14.972420 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afdfa6-5598-4c99-802e-408b17cfca99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:14.972889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afdfa6-5598-4c99-802e-408b17cfca99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:14.974423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2afdfa6-5598-4c99-802e-408b17cfca99 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:14.987923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 333/660] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:14.988967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afdfa6-5598-4c99-802e-408b17cfca99 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:15.042218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3c90f-dd2e-4058-b302-ca3ad4d19b3e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:15.088865 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdaccb-5489-4fa9-bfd5-d022f3817406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:15.090729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdaccb-5489-4fa9-bfd5-d022f3817406 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:15.091013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdaccb-5489-4fa9-bfd5-d022f3817406 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:15.091281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdaccb-5489-4fa9-bfd5-d022f3817406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:15.113816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:15.126228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2afdfa6-5598-4c99-802e-408b17cfca99 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:15.173915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b592236-f3ae-4043-85a6-cf24ce09491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:15.174520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b592236-f3ae-4043-85a6-cf24ce09491d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:15.174796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b592236-f3ae-4043-85a6-cf24ce09491d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:15.175092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b592236-f3ae-4043-85a6-cf24ce09491d None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:15.176062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b592236-f3ae-4043-85a6-cf24ce09491d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:15.176365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b592236-f3ae-4043-85a6-cf24ce09491d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:15.178674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b592236-f3ae-4043-85a6-cf24ce09491d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:15.179261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdaccb-5489-4fa9-bfd5-d022f3817406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:15.180882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdaccb-5489-4fa9-bfd5-d022f3817406 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:15.198334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:15.203043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 334/661] 10.222.0.25 () {56 vars in 1139 bytes} [Thu Sep 22 21:57:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:15.203775 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b592236-f3ae-4043-85a6-cf24ce09491d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:15.292228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 328/662] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:57:15.293529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff800f54-5e0c-4592-bcfe-c2ce9ec95815 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:15.293814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff800f54-5e0c-4592-bcfe-c2ce9ec95815 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:15.294090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff800f54-5e0c-4592-bcfe-c2ce9ec95815 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:15.299400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 329/663] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:15.299400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff800f54-5e0c-4592-bcfe-c2ce9ec95815 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:15.322982 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b592236-f3ae-4043-85a6-cf24ce09491d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:15.360536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63358312-a0c5-4105-b6b1-4893b778fc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:15.361414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358312-a0c5-4105-b6b1-4893b778fc18 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:15.361691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358312-a0c5-4105-b6b1-4893b778fc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:15.361961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63358312-a0c5-4105-b6b1-4893b778fc18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:15.368554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63358312-a0c5-4105-b6b1-4893b778fc18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:15.370208 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63358312-a0c5-4105-b6b1-4893b778fc18 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-63358312-a0c5-4105-b6b1-4893b778fc18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:15.381054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:15.382060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:15.382932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:15.382932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:15.382932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:15.382932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:15.382932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:15.382932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:15.382932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:15.382932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:15.386986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 335/664] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:15.387855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63358312-a0c5-4105-b6b1-4893b778fc18 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:15.398413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:15.406733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff800f54-5e0c-4592-bcfe-c2ce9ec95815 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:15.422109 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:15.443240 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c569b16-8b06-4ee2-aa35-54439dfd03df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:15.443830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c569b16-8b06-4ee2-aa35-54439dfd03df None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:15.444146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c569b16-8b06-4ee2-aa35-54439dfd03df None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:15.444428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c569b16-8b06-4ee2-aa35-54439dfd03df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:15.498321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63358312-a0c5-4105-b6b1-4893b778fc18 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:15.529098 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c569b16-8b06-4ee2-aa35-54439dfd03df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:15.531327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c569b16-8b06-4ee2-aa35-54439dfd03df None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:15.536399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdcb024-d4d3-44fe-8e98-2dd57f80f422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:15.537030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb024-d4d3-44fe-8e98-2dd57f80f422 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:15.537338 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb024-d4d3-44fe-8e98-2dd57f80f422 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:15.537612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdcb024-d4d3-44fe-8e98-2dd57f80f422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:15.548248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcb024-d4d3-44fe-8e98-2dd57f80f422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:15.549731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdcb024-d4d3-44fe-8e98-2dd57f80f422 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-efdcb024-d4d3-44fe-8e98-2dd57f80f422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:15.566417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:15.567370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:15.568256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:15.568256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:15.568256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:15.572228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 336/665] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:15.573904 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7df10-3e32-4c01-850f-247ae8194285 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:15.574191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7df10-3e32-4c01-850f-247ae8194285 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:15.574464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7df10-3e32-4c01-850f-247ae8194285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:15.592861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-31f7df10-3e32-4c01-850f-247ae8194285 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:15.594087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-31f7df10-3e32-4c01-850f-247ae8194285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:15.643166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 330/666] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:57:15.643166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c569b16-8b06-4ee2-aa35-54439dfd03df None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:15.659087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:15.660194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:15.681524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 337/667] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:15.682502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7df10-3e32-4c01-850f-247ae8194285 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:15.751327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c569b16-8b06-4ee2-aa35-54439dfd03df None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:15.799037 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef86772-f1a8-4a31-a9de-dcbe1fb65309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:15.799648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef86772-f1a8-4a31-a9de-dcbe1fb65309 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:15.799925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef86772-f1a8-4a31-a9de-dcbe1fb65309 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:15.800194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef86772-f1a8-4a31-a9de-dcbe1fb65309 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:15.800869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef86772-f1a8-4a31-a9de-dcbe1fb65309 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:15.801198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef86772-f1a8-4a31-a9de-dcbe1fb65309 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:15.802626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef86772-f1a8-4a31-a9de-dcbe1fb65309 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:15.814989 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 331/668] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:15.815891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef86772-f1a8-4a31-a9de-dcbe1fb65309 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:15.817335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7df10-3e32-4c01-850f-247ae8194285 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:15.863553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10da58a-1fcf-4438-9248-812c64bda0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:15.864209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10da58a-1fcf-4438-9248-812c64bda0fe None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:15.864488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10da58a-1fcf-4438-9248-812c64bda0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:15.864825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10da58a-1fcf-4438-9248-812c64bda0fe None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:15.865708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10da58a-1fcf-4438-9248-812c64bda0fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:15.866011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10da58a-1fcf-4438-9248-812c64bda0fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:15.867422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10da58a-1fcf-4438-9248-812c64bda0fe None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:15.879541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 338/669] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:15.880511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10da58a-1fcf-4438-9248-812c64bda0fe None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:15.908317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:15.935985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:15.949209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef86772-f1a8-4a31-a9de-dcbe1fb65309 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:15.985255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c35297-af6b-46d5-ae72-70e0a3e4bd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:15.985878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c35297-af6b-46d5-ae72-70e0a3e4bd55 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:15.986156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c35297-af6b-46d5-ae72-70e0a3e4bd55 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:15.986431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c35297-af6b-46d5-ae72-70e0a3e4bd55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:15.991065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10da58a-1fcf-4438-9248-812c64bda0fe None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:15.998277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c35297-af6b-46d5-ae72-70e0a3e4bd55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:15.999733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c35297-af6b-46d5-ae72-70e0a3e4bd55 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-84c35297-af6b-46d5-ae72-70e0a3e4bd55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:16.014342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:16.015319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:16.016227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:16.016227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:16.016227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:16.016227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 332/670] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:16.016227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c35297-af6b-46d5-ae72-70e0a3e4bd55 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:16.034567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46411e3-1570-4154-9993-0fa1e64ad5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:16.035176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46411e3-1570-4154-9993-0fa1e64ad5cf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:16.035461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46411e3-1570-4154-9993-0fa1e64ad5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:16.035725 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46411e3-1570-4154-9993-0fa1e64ad5cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:16.036569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46411e3-1570-4154-9993-0fa1e64ad5cf None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=d011e54e12674eaeb82c68da628ecb9d) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:16.037012 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46411e3-1570-4154-9993-0fa1e64ad5cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:16.038484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46411e3-1570-4154-9993-0fa1e64ad5cf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:16.044977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 339/671] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:15 2022] GET /identity/v3/role_assignments?user.id=d011e54e12674eaeb82c68da628ecb9d&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:16.045804 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46411e3-1570-4154-9993-0fa1e64ad5cf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:16.151499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c35297-af6b-46d5-ae72-70e0a3e4bd55 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:16.180941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:16.192231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76590de2-fdbe-4764-8d49-e2f0818362a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:16.193282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590de2-fdbe-4764-8d49-e2f0818362a3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:16.193746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590de2-fdbe-4764-8d49-e2f0818362a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:16.194229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76590de2-fdbe-4764-8d49-e2f0818362a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:16.196995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46411e3-1570-4154-9993-0fa1e64ad5cf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:16.202992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76590de2-fdbe-4764-8d49-e2f0818362a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:16.204852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76590de2-fdbe-4764-8d49-e2f0818362a3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:16.216094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-76590de2-fdbe-4764-8d49-e2f0818362a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:16.220853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:16.221821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:16.222730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:16.222730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:16.222730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:16.222730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:16.222730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:16.222730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:16.222730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:16.222730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:16.225316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 333/672] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:16.226331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76590de2-fdbe-4764-8d49-e2f0818362a3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:16.247385 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d3e88-b528-4b8b-a44b-74a58e7c3e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:16.247950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d3e88-b528-4b8b-a44b-74a58e7c3e4e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:16.248850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d3e88-b528-4b8b-a44b-74a58e7c3e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:16.248850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d3e88-b528-4b8b-a44b-74a58e7c3e4e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:16.263209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d3e88-b528-4b8b-a44b-74a58e7c3e4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ce3d3e88-b528-4b8b-a44b-74a58e7c3e4e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:16.269689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:16.270649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:16.271494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:16.273816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 340/673] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:16.274745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d3e88-b528-4b8b-a44b-74a58e7c3e4e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:16.359459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76590de2-fdbe-4764-8d49-e2f0818362a3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:16.400107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d3e88-b528-4b8b-a44b-74a58e7c3e4e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:16.407831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218e388-25f7-46c1-91bd-1db72611c61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:16.409502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218e388-25f7-46c1-91bd-1db72611c61d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:16.409931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218e388-25f7-46c1-91bd-1db72611c61d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:16.410369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218e388-25f7-46c1-91bd-1db72611c61d None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:16.411640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218e388-25f7-46c1-91bd-1db72611c61d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:16.412097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218e388-25f7-46c1-91bd-1db72611c61d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:16.414355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218e388-25f7-46c1-91bd-1db72611c61d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:16.432910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 334/674] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:16.434617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218e388-25f7-46c1-91bd-1db72611c61d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:16.440024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c409f3-f4af-4983-aa67-c1cb80a37d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:16.440755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c409f3-f4af-4983-aa67-c1cb80a37d79 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:16.441152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c409f3-f4af-4983-aa67-c1cb80a37d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:16.441531 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c409f3-f4af-4983-aa67-c1cb80a37d79 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:16.442318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c409f3-f4af-4983-aa67-c1cb80a37d79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:16.442722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c409f3-f4af-4983-aa67-c1cb80a37d79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:16.444250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c409f3-f4af-4983-aa67-c1cb80a37d79 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:16.460191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 341/675] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:16.461324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c409f3-f4af-4983-aa67-c1cb80a37d79 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:16.497328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:16.503332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:16.564729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218e388-25f7-46c1-91bd-1db72611c61d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:16.581944 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c409f3-f4af-4983-aa67-c1cb80a37d79 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:16.601598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42513280-ebe6-4d22-b6d4-fbe1ba06335f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:16.602444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42513280-ebe6-4d22-b6d4-fbe1ba06335f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:16.602870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42513280-ebe6-4d22-b6d4-fbe1ba06335f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:16.603365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42513280-ebe6-4d22-b6d4-fbe1ba06335f None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:16.604176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42513280-ebe6-4d22-b6d4-fbe1ba06335f None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:16.604637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42513280-ebe6-4d22-b6d4-fbe1ba06335f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:16.606187 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42513280-ebe6-4d22-b6d4-fbe1ba06335f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:16.615032 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 335/676] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:16.616059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42513280-ebe6-4d22-b6d4-fbe1ba06335f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:16.622187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc0c8c-8360-4ce9-a63a-ef3794a3a65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:16.622949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc0c8c-8360-4ce9-a63a-ef3794a3a65b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:16.623324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc0c8c-8360-4ce9-a63a-ef3794a3a65b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:16.623707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc0c8c-8360-4ce9-a63a-ef3794a3a65b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:16.631647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc0c8c-8360-4ce9-a63a-ef3794a3a65b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:16.633682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efc0c8c-8360-4ce9-a63a-ef3794a3a65b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5efc0c8c-8360-4ce9-a63a-ef3794a3a65b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:16.646223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:16.647324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:16.648436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:16.648436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:16.648436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:16.648436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:16.648436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:16.648436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:16.648436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:16.648436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:16.650359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 342/677] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:16.651731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc0c8c-8360-4ce9-a63a-ef3794a3a65b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:16.725059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42513280-ebe6-4d22-b6d4-fbe1ba06335f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:16.734762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:16.758081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:16.761577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94bfa3-0b72-4183-9246-cf695f490e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:16.762188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94bfa3-0b72-4183-9246-cf695f490e8e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:16.762459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94bfa3-0b72-4183-9246-cf695f490e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:16.762732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94bfa3-0b72-4183-9246-cf695f490e8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:16.763447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94bfa3-0b72-4183-9246-cf695f490e8e None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=fcfdbb6b3dd04478ae99681cc649fdb9) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:16.763756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94bfa3-0b72-4183-9246-cf695f490e8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:16.765266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94bfa3-0b72-4183-9246-cf695f490e8e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:16.768192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc0c8c-8360-4ce9-a63a-ef3794a3a65b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:16.771720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 336/678] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:16 2022] GET /identity/v3/role_assignments?user.id=fcfdbb6b3dd04478ae99681cc649fdb9&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:16.772532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94bfa3-0b72-4183-9246-cf695f490e8e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:16.817268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1715bc3-280f-4ebf-8aeb-6bf2e3f6efd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:16.818336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1715bc3-280f-4ebf-8aeb-6bf2e3f6efd4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:16.818814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1715bc3-280f-4ebf-8aeb-6bf2e3f6efd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:16.819295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1715bc3-280f-4ebf-8aeb-6bf2e3f6efd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:16.832099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1715bc3-280f-4ebf-8aeb-6bf2e3f6efd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:16.834128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1715bc3-280f-4ebf-8aeb-6bf2e3f6efd4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f1715bc3-280f-4ebf-8aeb-6bf2e3f6efd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:16.852242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:16.853334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:16.854969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:16.854969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:16.854969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:16.857442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 343/679] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:16.862639 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1715bc3-280f-4ebf-8aeb-6bf2e3f6efd4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:16.979679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94bfa3-0b72-4183-9246-cf695f490e8e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:17.031364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef98f3d-c12c-4c83-a106-1d0aa28fe605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:17.032581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef98f3d-c12c-4c83-a106-1d0aa28fe605 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:17.033065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef98f3d-c12c-4c83-a106-1d0aa28fe605 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:17.034013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef98f3d-c12c-4c83-a106-1d0aa28fe605 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:17.035220 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef98f3d-c12c-4c83-a106-1d0aa28fe605 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:17.035965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef98f3d-c12c-4c83-a106-1d0aa28fe605 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:17.038363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef98f3d-c12c-4c83-a106-1d0aa28fe605 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:17.042386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1715bc3-280f-4ebf-8aeb-6bf2e3f6efd4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:17.059272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 337/680] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:17.064628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef98f3d-c12c-4c83-a106-1d0aa28fe605 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:17.094394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952ecbac-b4e5-44ed-8f4a-96184e3dd119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:17.095115 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecbac-b4e5-44ed-8f4a-96184e3dd119 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:17.095393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecbac-b4e5-44ed-8f4a-96184e3dd119 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:17.096490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952ecbac-b4e5-44ed-8f4a-96184e3dd119 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:17.097165 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ecbac-b4e5-44ed-8f4a-96184e3dd119 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:17.097474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ecbac-b4e5-44ed-8f4a-96184e3dd119 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:17.099118 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952ecbac-b4e5-44ed-8f4a-96184e3dd119 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:17.099761 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:17.118672 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 344/681] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:17.121332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d11b8-a8c6-4ffd-b31b-f7885400c88c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:17.121692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d11b8-a8c6-4ffd-b31b-f7885400c88c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:17.121986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d11b8-a8c6-4ffd-b31b-f7885400c88c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:17.123219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 345/682] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:17.125998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d11b8-a8c6-4ffd-b31b-f7885400c88c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:17.147943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:17.184863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef98f3d-c12c-4c83-a106-1d0aa28fe605 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:17.220786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d1f5b-8bdc-4018-bed7-1bf07c6b39ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:17.221418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d1f5b-8bdc-4018-bed7-1bf07c6b39ce None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:17.221810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d1f5b-8bdc-4018-bed7-1bf07c6b39ce None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:17.222090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d1f5b-8bdc-4018-bed7-1bf07c6b39ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:17.222805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d1f5b-8bdc-4018-bed7-1bf07c6b39ce None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=18c956fa035f40fa88fbf662541121ab) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:17.223113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d1f5b-8bdc-4018-bed7-1bf07c6b39ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:17.224649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d1f5b-8bdc-4018-bed7-1bf07c6b39ce None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:17.232061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 338/683] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:17 2022] GET /identity/v3/role_assignments?user.id=18c956fa035f40fa88fbf662541121ab&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:17.233428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b042e7-b650-4c37-977b-7871c76995ec None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:17.233718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b042e7-b650-4c37-977b-7871c76995ec None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:17.233987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b042e7-b650-4c37-977b-7871c76995ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:17.255640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-65b042e7-b650-4c37-977b-7871c76995ec None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:17.257663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-65b042e7-b650-4c37-977b-7871c76995ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:17.263060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d11b8-a8c6-4ffd-b31b-f7885400c88c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:17.345447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423b084-087d-45df-a2b6-ef7a5fb4ebf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:17.346345 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423b084-087d-45df-a2b6-ef7a5fb4ebf7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:17.346769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423b084-087d-45df-a2b6-ef7a5fb4ebf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:17.347210 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423b084-087d-45df-a2b6-ef7a5fb4ebf7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:17.349256 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 339/684] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:17.366349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423b084-087d-45df-a2b6-ef7a5fb4ebf7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:17.368082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423b084-087d-45df-a2b6-ef7a5fb4ebf7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a423b084-087d-45df-a2b6-ef7a5fb4ebf7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:17.390997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:17.392205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:17.393412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:17.393412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:17.393412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:17.393412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:17.393412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:17.393412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:17.393412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:17.393412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:17.393412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:17.393412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:17.395650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 346/685] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:17.401834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b042e7-b650-4c37-977b-7871c76995ec None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:17.450670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:17.530195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b042e7-b650-4c37-977b-7871c76995ec None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:17.568819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f44b42-5f02-40f6-be43-7e34ff825fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:17.569465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f44b42-5f02-40f6-be43-7e34ff825fed None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:17.569744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f44b42-5f02-40f6-be43-7e34ff825fed None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:17.570020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f44b42-5f02-40f6-be43-7e34ff825fed None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:17.570752 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f44b42-5f02-40f6-be43-7e34ff825fed None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:17.571060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f44b42-5f02-40f6-be43-7e34ff825fed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:17.572526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f44b42-5f02-40f6-be43-7e34ff825fed None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:17.591323 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 340/686] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 22 21:57:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:17.596483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423b084-087d-45df-a2b6-ef7a5fb4ebf7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:17.664715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:17.698204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f44b42-5f02-40f6-be43-7e34ff825fed None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:17.713268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423b084-087d-45df-a2b6-ef7a5fb4ebf7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:17.755805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be26f0f-dfe2-4d85-851f-2503959b4fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:17.756437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be26f0f-dfe2-4d85-851f-2503959b4fdd None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:17.756714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be26f0f-dfe2-4d85-851f-2503959b4fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:17.756984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be26f0f-dfe2-4d85-851f-2503959b4fdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:17.818833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:17.841318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f44b42-5f02-40f6-be43-7e34ff825fed None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:17.855396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be26f0f-dfe2-4d85-851f-2503959b4fdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:17.856859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be26f0f-dfe2-4d85-851f-2503959b4fdd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:17.912081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599a5be-d4fe-4365-abcf-42f4aaab39f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:17.912707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a5be-d4fe-4365-abcf-42f4aaab39f3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:17.912982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a5be-d4fe-4365-abcf-42f4aaab39f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:17.915324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3599a5be-d4fe-4365-abcf-42f4aaab39f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:17.937852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599a5be-d4fe-4365-abcf-42f4aaab39f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:17.940509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3599a5be-d4fe-4365-abcf-42f4aaab39f3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-3599a5be-d4fe-4365-abcf-42f4aaab39f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:17.969955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:17.970958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:17.971845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:17.971845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:17.971845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:17.976837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 341/687] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:17.981171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599a5be-d4fe-4365-abcf-42f4aaab39f3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:17.986439 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:18.038050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 347/688] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 22 21:57:18.044109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be26f0f-dfe2-4d85-851f-2503959b4fdd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:18.190404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3599a5be-d4fe-4365-abcf-42f4aaab39f3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:18.213340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be26f0f-dfe2-4d85-851f-2503959b4fdd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:18.257710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272cd85-f041-4382-a535-7397ba753b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:18.258312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272cd85-f041-4382-a535-7397ba753b62 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:18.258587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272cd85-f041-4382-a535-7397ba753b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:18.258856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d272cd85-f041-4382-a535-7397ba753b62 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:18.259526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272cd85-f041-4382-a535-7397ba753b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:18.259828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272cd85-f041-4382-a535-7397ba753b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:18.261974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d272cd85-f041-4382-a535-7397ba753b62 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:18.265042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:18.277881 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696b619-f538-447a-bdd2-8e0993e68233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:18.278546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696b619-f538-447a-bdd2-8e0993e68233 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:18.278840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696b619-f538-447a-bdd2-8e0993e68233 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:18.279032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696b619-f538-447a-bdd2-8e0993e68233 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:18.294356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 342/689] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:18.297866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272cd85-f041-4382-a535-7397ba753b62 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:18.302949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696b619-f538-447a-bdd2-8e0993e68233 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:18.308596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8696b619-f538-447a-bdd2-8e0993e68233 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-8696b619-f538-447a-bdd2-8e0993e68233 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:18.338835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:18.339823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:18.340973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:18.340973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:18.340973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:18.343492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 348/690] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:18.350007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696b619-f538-447a-bdd2-8e0993e68233 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:18.410908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:18.432543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d272cd85-f041-4382-a535-7397ba753b62 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:18.475242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c198679-f2e8-495b-918a-c51a478a77ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:18.476619 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c198679-f2e8-495b-918a-c51a478a77ba None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:18.477143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c198679-f2e8-495b-918a-c51a478a77ba None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:18.477650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c198679-f2e8-495b-918a-c51a478a77ba None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:18.478621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c198679-f2e8-495b-918a-c51a478a77ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:18.480542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c198679-f2e8-495b-918a-c51a478a77ba None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:18.523846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 343/691] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:57:18 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:57:18.564059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e679e79-4e01-4519-8cb0-01912af7de1a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:18.564727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e679e79-4e01-4519-8cb0-01912af7de1a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:18.565239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e679e79-4e01-4519-8cb0-01912af7de1a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:18.566905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 344/692] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:18.572508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980e6fa-8c9d-4ddc-85a8-6c8105fd259c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:18.573201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980e6fa-8c9d-4ddc-85a8-6c8105fd259c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:18.573713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980e6fa-8c9d-4ddc-85a8-6c8105fd259c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:18.604088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4980e6fa-8c9d-4ddc-85a8-6c8105fd259c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:18.607627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4980e6fa-8c9d-4ddc-85a8-6c8105fd259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:18.616303 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8696b619-f538-447a-bdd2-8e0993e68233 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:18.705139 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283152e1-ae92-48a0-b22f-970a2f267b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:18.705790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283152e1-ae92-48a0-b22f-970a2f267b15 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:18.706068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283152e1-ae92-48a0-b22f-970a2f267b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:18.707540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283152e1-ae92-48a0-b22f-970a2f267b15 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:18.707540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283152e1-ae92-48a0-b22f-970a2f267b15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:18.707540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283152e1-ae92-48a0-b22f-970a2f267b15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:18.708946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283152e1-ae92-48a0-b22f-970a2f267b15 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:18.725826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:18.730930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 349/693] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:18.739436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283152e1-ae92-48a0-b22f-970a2f267b15 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:18.765045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 345/694] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:57:18.823030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:18.849925 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283152e1-ae92-48a0-b22f-970a2f267b15 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:18.894762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e5026-6f07-4b2a-b19a-ae927e9f9e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:18.895779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e5026-6f07-4b2a-b19a-ae927e9f9e0f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:18.896260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e5026-6f07-4b2a-b19a-ae927e9f9e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:18.896730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32e5026-6f07-4b2a-b19a-ae927e9f9e0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:18.905619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e5026-6f07-4b2a-b19a-ae927e9f9e0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:18.907338 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32e5026-6f07-4b2a-b19a-ae927e9f9e0f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a32e5026-6f07-4b2a-b19a-ae927e9f9e0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:18.920754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:18.921751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:18.922662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:18.922662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:18.922662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:18.922662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:18.922662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:18.922662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:18.922662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:18.922662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:18.925080 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 350/695] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:18.931027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980e6fa-8c9d-4ddc-85a8-6c8105fd259c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:19.061451 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980e6fa-8c9d-4ddc-85a8-6c8105fd259c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:19.078852 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e5026-6f07-4b2a-b19a-ae927e9f9e0f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:19.107575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba140d-37af-4a1a-8207-cfd4eb6b4710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:19.108205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba140d-37af-4a1a-8207-cfd4eb6b4710 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:19.108465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba140d-37af-4a1a-8207-cfd4eb6b4710 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:19.108738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba140d-37af-4a1a-8207-cfd4eb6b4710 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:19.109456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba140d-37af-4a1a-8207-cfd4eb6b4710 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:19.109758 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba140d-37af-4a1a-8207-cfd4eb6b4710 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:19.111177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba140d-37af-4a1a-8207-cfd4eb6b4710 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:19.124206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 346/696] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:19.134263 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba140d-37af-4a1a-8207-cfd4eb6b4710 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:19.216221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:19.282979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32e5026-6f07-4b2a-b19a-ae927e9f9e0f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:19.332145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba140d-37af-4a1a-8207-cfd4eb6b4710 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:19.333467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e3357-5653-43b1-b6e4-8dc231d424a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:19.335722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e3357-5653-43b1-b6e4-8dc231d424a1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:19.336217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e3357-5653-43b1-b6e4-8dc231d424a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:19.336762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e3357-5653-43b1-b6e4-8dc231d424a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:19.354923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:19.361631 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e3357-5653-43b1-b6e4-8dc231d424a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-113e3357-5653-43b1-b6e4-8dc231d424a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:19.367253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:19.368209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:19.369554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:19.372576 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 351/697] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:19.378816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e3357-5653-43b1-b6e4-8dc231d424a1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:19.407539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cceb6d-e7ce-4a02-93ad-a8a35a9f3f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:19.409858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cceb6d-e7ce-4a02-93ad-a8a35a9f3f3a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:19.410373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cceb6d-e7ce-4a02-93ad-a8a35a9f3f3a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:19.410864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cceb6d-e7ce-4a02-93ad-a8a35a9f3f3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:19.411812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cceb6d-e7ce-4a02-93ad-a8a35a9f3f3a None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=471e7550ae7a4f909851215d3ed148c6) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:19.412377 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cceb6d-e7ce-4a02-93ad-a8a35a9f3f3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:19.414038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cceb6d-e7ce-4a02-93ad-a8a35a9f3f3a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:19.421767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 347/698] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:19 2022] GET /identity/v3/role_assignments?user.id=471e7550ae7a4f909851215d3ed148c6&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:19.430334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1862f2d-935d-43ef-a5fc-09a5c165180e None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:19.430951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1862f2d-935d-43ef-a5fc-09a5c165180e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:19.431506 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1862f2d-935d-43ef-a5fc-09a5c165180e None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:19.433869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 348/699] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:19.443045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635dc07-ee21-444d-9829-e8ba9138e378 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:19.443669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635dc07-ee21-444d-9829-e8ba9138e378 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:19.444151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f635dc07-ee21-444d-9829-e8ba9138e378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:19.482304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f635dc07-ee21-444d-9829-e8ba9138e378 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:19.484371 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f635dc07-ee21-444d-9829-e8ba9138e378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:19.508004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e3357-5653-43b1-b6e4-8dc231d424a1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:19.547426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83b1e6-e12d-4c6f-9bd9-74b11cf51c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:19.548042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83b1e6-e12d-4c6f-9bd9-74b11cf51c65 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:19.548328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83b1e6-e12d-4c6f-9bd9-74b11cf51c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:19.548592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f83b1e6-e12d-4c6f-9bd9-74b11cf51c65 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:19.549300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83b1e6-e12d-4c6f-9bd9-74b11cf51c65 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:19.549648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83b1e6-e12d-4c6f-9bd9-74b11cf51c65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:19.551210 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f83b1e6-e12d-4c6f-9bd9-74b11cf51c65 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:19.562500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 352/700] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:57:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:19.570213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83b1e6-e12d-4c6f-9bd9-74b11cf51c65 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:19.625551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 349/701] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 22 21:57:19.665427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:19.728301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f83b1e6-e12d-4c6f-9bd9-74b11cf51c65 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:19.762884 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c3a81-2fa8-42ae-8d0f-97e81ef60be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:19.763508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c3a81-2fa8-42ae-8d0f-97e81ef60be9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:19.763833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c3a81-2fa8-42ae-8d0f-97e81ef60be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:19.764108 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c3a81-2fa8-42ae-8d0f-97e81ef60be9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:19.775985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c3a81-2fa8-42ae-8d0f-97e81ef60be9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:19.777561 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c3a81-2fa8-42ae-8d0f-97e81ef60be9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cc1c3a81-2fa8-42ae-8d0f-97e81ef60be9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:19.796121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:19.797164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:19.798084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:19.798084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:19.798084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:19.798084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 353/702] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:19.802708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635dc07-ee21-444d-9829-e8ba9138e378 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:19.840868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:19.912241 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f635dc07-ee21-444d-9829-e8ba9138e378 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:19.946572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed9919-a600-4ca3-a3fb-0f240ac95377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:19.947773 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed9919-a600-4ca3-a3fb-0f240ac95377 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:19.948051 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed9919-a600-4ca3-a3fb-0f240ac95377 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:19.948339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed9919-a600-4ca3-a3fb-0f240ac95377 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:19.948939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed9919-a600-4ca3-a3fb-0f240ac95377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:19.949265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed9919-a600-4ca3-a3fb-0f240ac95377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:19.950694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed9919-a600-4ca3-a3fb-0f240ac95377 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:19.963744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 350/703] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:19.969043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c3a81-2fa8-42ae-8d0f-97e81ef60be9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:19.978075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed9919-a600-4ca3-a3fb-0f240ac95377 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:20.102098 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:20.109775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed9919-a600-4ca3-a3fb-0f240ac95377 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:20.113411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c3a81-2fa8-42ae-8d0f-97e81ef60be9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:20.129384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:20.149865 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae8e5c-0375-4d97-8938-19141c5f5d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:20.150861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae8e5c-0375-4d97-8938-19141c5f5d7f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:20.151143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae8e5c-0375-4d97-8938-19141c5f5d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:20.151417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edae8e5c-0375-4d97-8938-19141c5f5d7f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:20.152006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae8e5c-0375-4d97-8938-19141c5f5d7f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:20.153888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edae8e5c-0375-4d97-8938-19141c5f5d7f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:20.159167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3997020a-0168-40b8-a82d-51f5e89d5a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:20.160070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997020a-0168-40b8-a82d-51f5e89d5a93 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:20.160554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997020a-0168-40b8-a82d-51f5e89d5a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:20.161023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3997020a-0168-40b8-a82d-51f5e89d5a93 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-edae8e5c-0375-4d97-8938-19141c5f5d7f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:20.166563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:20.167703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:20.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 351/704] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 22 21:57:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:20.174080 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997020a-0168-40b8-a82d-51f5e89d5a93 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:20.176123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3997020a-0168-40b8-a82d-51f5e89d5a93 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:20.178307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae8e5c-0375-4d97-8938-19141c5f5d7f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-3997020a-0168-40b8-a82d-51f5e89d5a93 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:20.192304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:20.193356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:20.194302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:20.194302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:20.194302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:20.194302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:20.194302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:20.194302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:20.194302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:20.194302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:20.194302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:57:20.194302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:20.194302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 354/705] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 22 21:57:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:20.195369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188e49-f645-4e51-bf7a-316ca99d776f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:20.195917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188e49-f645-4e51-bf7a-316ca99d776f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:20.196326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86188e49-f645-4e51-bf7a-316ca99d776f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:20.198419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 355/706] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:20.199564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86188e49-f645-4e51-bf7a-316ca99d776f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:20.300836 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edae8e5c-0375-4d97-8938-19141c5f5d7f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:20.315487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86188e49-f645-4e51-bf7a-316ca99d776f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:20.339944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab483fd-83e4-4853-9bf5-c3acb196c204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:20.341068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab483fd-83e4-4853-9bf5-c3acb196c204 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:20.341513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab483fd-83e4-4853-9bf5-c3acb196c204 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:20.342019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab483fd-83e4-4853-9bf5-c3acb196c204 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:20.342928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab483fd-83e4-4853-9bf5-c3acb196c204 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:20.343378 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab483fd-83e4-4853-9bf5-c3acb196c204 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:20.345022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab483fd-83e4-4853-9bf5-c3acb196c204 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:20.350395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12155640-2680-47a5-97b3-ef146d08d821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:20.355909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12155640-2680-47a5-97b3-ef146d08d821 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:20.355909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12155640-2680-47a5-97b3-ef146d08d821 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:20.355909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12155640-2680-47a5-97b3-ef146d08d821 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:20.355909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12155640-2680-47a5-97b3-ef146d08d821 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:20.355909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12155640-2680-47a5-97b3-ef146d08d821 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:20.355909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12155640-2680-47a5-97b3-ef146d08d821 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:20.356759 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 352/707] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 22 21:57:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:20.358330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3124c-80d8-4e5c-b0c8-732fd410e55f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:20.358754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3124c-80d8-4e5c-b0c8-732fd410e55f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:20.359166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3124c-80d8-4e5c-b0c8-732fd410e55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:20.374033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 356/708] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 22 21:57:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:20.374947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12155640-2680-47a5-97b3-ef146d08d821 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:20.379469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-99d3124c-80d8-4e5c-b0c8-732fd410e55f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:20.380868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-99d3124c-80d8-4e5c-b0c8-732fd410e55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:20.401662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:20.406301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:20.460543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 353/709] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:20.461756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3124c-80d8-4e5c-b0c8-732fd410e55f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:20.477973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12155640-2680-47a5-97b3-ef146d08d821 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:20.514900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d39f0e-7503-4d48-92c8-376d9fde8b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:20.515116 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d39f0e-7503-4d48-92c8-376d9fde8b42 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:20.515365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d39f0e-7503-4d48-92c8-376d9fde8b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:20.515623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d39f0e-7503-4d48-92c8-376d9fde8b42 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:20.516249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d39f0e-7503-4d48-92c8-376d9fde8b42 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:20.516548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d39f0e-7503-4d48-92c8-376d9fde8b42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:20.517936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d39f0e-7503-4d48-92c8-376d9fde8b42 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:20.529248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 357/710] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 22 21:57:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:20.585336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3124c-80d8-4e5c-b0c8-732fd410e55f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:20.626405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff9cea-a2c2-4382-9252-c0ac90d6339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:20.627334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff9cea-a2c2-4382-9252-c0ac90d6339a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:20.627751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff9cea-a2c2-4382-9252-c0ac90d6339a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:20.628150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fff9cea-a2c2-4382-9252-c0ac90d6339a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:20.657242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:20.710359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff9cea-a2c2-4382-9252-c0ac90d6339a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:20.717868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fff9cea-a2c2-4382-9252-c0ac90d6339a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:20.825235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 354/711] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:57:20.834784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d39f0e-7503-4d48-92c8-376d9fde8b42 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:20.859827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff9cea-a2c2-4382-9252-c0ac90d6339a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:20.953467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:20.983908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:20.993154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d39f0e-7503-4d48-92c8-376d9fde8b42 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:21.037098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f3fb3-fb46-4adf-8a0a-b3a58d57761c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:21.037770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f3fb3-fb46-4adf-8a0a-b3a58d57761c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:21.038083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f3fb3-fb46-4adf-8a0a-b3a58d57761c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:21.038314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88f3fb3-fb46-4adf-8a0a-b3a58d57761c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:21.044329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fff9cea-a2c2-4382-9252-c0ac90d6339a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:21.076911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f3fb3-fb46-4adf-8a0a-b3a58d57761c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:21.078412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88f3fb3-fb46-4adf-8a0a-b3a58d57761c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a88f3fb3-fb46-4adf-8a0a-b3a58d57761c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:21.098619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:21.099600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:21.100472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:21.100472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:21.100472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:21.100472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 358/712] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:21.109551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f3fb3-fb46-4adf-8a0a-b3a58d57761c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:21.158414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338fb02-5382-4b63-8d1c-026b3d14e1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:21.159058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338fb02-5382-4b63-8d1c-026b3d14e1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:21.159333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338fb02-5382-4b63-8d1c-026b3d14e1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:21.159603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9338fb02-5382-4b63-8d1c-026b3d14e1e8 None admin] PATH_INFO: `/v3/services/network` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:21.160190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338fb02-5382-4b63-8d1c-026b3d14e1e8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:21.161580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9338fb02-5382-4b63-8d1c-026b3d14e1e8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-9338fb02-5382-4b63-8d1c-026b3d14e1e8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:21.190652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:21.191717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 21:57:21.192635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:21.196427 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 355/713] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:20 2022] GET /identity/v3/services/network => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:21.199142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338fb02-5382-4b63-8d1c-026b3d14e1e8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:21.227220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88f3fb3-fb46-4adf-8a0a-b3a58d57761c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:21.262056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc76c2-21a7-4b9c-a1a8-9218dbdcae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:21.262667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc76c2-21a7-4b9c-a1a8-9218dbdcae18 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:21.262947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc76c2-21a7-4b9c-a1a8-9218dbdcae18 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:21.263222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc76c2-21a7-4b9c-a1a8-9218dbdcae18 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:21.263862 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc76c2-21a7-4b9c-a1a8-9218dbdcae18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:21.264164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc76c2-21a7-4b9c-a1a8-9218dbdcae18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:21.265567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc76c2-21a7-4b9c-a1a8-9218dbdcae18 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:21.278493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 359/714] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:21.283834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc76c2-21a7-4b9c-a1a8-9218dbdcae18 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:21.298537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:21.391592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9338fb02-5382-4b63-8d1c-026b3d14e1e8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:21.392654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc76c2-21a7-4b9c-a1a8-9218dbdcae18 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:21.411080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:21.432254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6318a08-044b-4009-b2a6-318927179d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:21.433182 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6318a08-044b-4009-b2a6-318927179d3a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:21.433640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6318a08-044b-4009-b2a6-318927179d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:21.434110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6318a08-044b-4009-b2a6-318927179d3a None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:21.434917 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6318a08-044b-4009-b2a6-318927179d3a None admin] RBAC: Adding query filter params (name=network) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:21.435416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6318a08-044b-4009-b2a6-318927179d3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:21.436061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7a2c6-9b57-4829-90a8-bc47daad0d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:21.436695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7a2c6-9b57-4829-90a8-bc47daad0d0a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:21.436972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7a2c6-9b57-4829-90a8-bc47daad0d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:21.437276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f7a2c6-9b57-4829-90a8-bc47daad0d0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:21.437795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6318a08-044b-4009-b2a6-318927179d3a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:21.447927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7a2c6-9b57-4829-90a8-bc47daad0d0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:21.447927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f7a2c6-9b57-4829-90a8-bc47daad0d0a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:21.448268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 356/715] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:57:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:21.449820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b595a9e-e641-4684-870a-b132b7cc8b98 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:21.450305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b595a9e-e641-4684-870a-b132b7cc8b98 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:21.450749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b595a9e-e641-4684-870a-b132b7cc8b98 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:21.452425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 357/716] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:21.453315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b595a9e-e641-4684-870a-b132b7cc8b98 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-83f7a2c6-9b57-4829-90a8-bc47daad0d0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:21.459271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:21.460231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:21.461234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:21.461234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:21.461234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:21.461234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:21.461234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:21.461234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:21.461234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:21.461234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:21.461234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 360/717] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:21.466895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7a2c6-9b57-4829-90a8-bc47daad0d0a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:21.542177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:21.573233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b595a9e-e641-4684-870a-b132b7cc8b98 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:21.583535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f7a2c6-9b57-4829-90a8-bc47daad0d0a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:21.610633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8212a6-b5ed-49da-b492-b6aaedce5ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:21.612058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8212a6-b5ed-49da-b492-b6aaedce5ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:21.612584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8212a6-b5ed-49da-b492-b6aaedce5ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:21.612997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8212a6-b5ed-49da-b492-b6aaedce5ac1 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:21.613838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8212a6-b5ed-49da-b492-b6aaedce5ac1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:21.614279 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8212a6-b5ed-49da-b492-b6aaedce5ac1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:21.616008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8212a6-b5ed-49da-b492-b6aaedce5ac1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:21.621514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67fa6b-6d25-4d6b-945d-0ff69cf717f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:21.622117 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67fa6b-6d25-4d6b-945d-0ff69cf717f9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:21.622396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67fa6b-6d25-4d6b-945d-0ff69cf717f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:21.622670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67fa6b-6d25-4d6b-945d-0ff69cf717f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:21.623369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67fa6b-6d25-4d6b-945d-0ff69cf717f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:21.623661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67fa6b-6d25-4d6b-945d-0ff69cf717f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:21.629176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67fa6b-6d25-4d6b-945d-0ff69cf717f9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:21.629717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 358/718] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:57:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:21.631345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342a18c-5c04-48c9-9fe2-d8994b81033d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:21.631829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342a18c-5c04-48c9-9fe2-d8994b81033d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:21.632247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342a18c-5c04-48c9-9fe2-d8994b81033d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:21.641213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 361/719] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:21.644729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67fa6b-6d25-4d6b-945d-0ff69cf717f9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:21.661589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f342a18c-5c04-48c9-9fe2-d8994b81033d None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:21.663036 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f342a18c-5c04-48c9-9fe2-d8994b81033d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:21.747183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:21.767289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67fa6b-6d25-4d6b-945d-0ff69cf717f9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:21.796662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:21.802879 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 359/720] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:21.805885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0471b-21e4-42a7-9baa-0ad183c7e772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:21.806536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0471b-21e4-42a7-9baa-0ad183c7e772 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:21.806816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0471b-21e4-42a7-9baa-0ad183c7e772 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:21.807195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd0471b-21e4-42a7-9baa-0ad183c7e772 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:21.807948 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0471b-21e4-42a7-9baa-0ad183c7e772 None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=d011e54e12674eaeb82c68da628ecb9d) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:21.808263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0471b-21e4-42a7-9baa-0ad183c7e772 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:21.809747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd0471b-21e4-42a7-9baa-0ad183c7e772 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:21.816434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 362/721] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:21 2022] GET /identity/v3/role_assignments?user.id=d011e54e12674eaeb82c68da628ecb9d&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:22.139489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342a18c-5c04-48c9-9fe2-d8994b81033d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:22.278814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342a18c-5c04-48c9-9fe2-d8994b81033d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:22.328565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7b2fd-e997-4f8d-98ac-3d5247373606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:22.329580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7b2fd-e997-4f8d-98ac-3d5247373606 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:22.330056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7b2fd-e997-4f8d-98ac-3d5247373606 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:22.330523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7b2fd-e997-4f8d-98ac-3d5247373606 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:22.346097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d7b2fd-e997-4f8d-98ac-3d5247373606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-82d7b2fd-e997-4f8d-98ac-3d5247373606 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:22.351364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:22.352338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:22.353838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:22.356247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 360/722] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:22.362276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0471b-21e4-42a7-9baa-0ad183c7e772 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:22.425028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc3d6f-0636-4a65-801b-20eff677694b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:22.425871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc3d6f-0636-4a65-801b-20eff677694b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:22.426349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc3d6f-0636-4a65-801b-20eff677694b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:22.428184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 361/723] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:22.439919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2d686-0833-4168-928b-e399a499b423 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:22.440193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2d686-0833-4168-928b-e399a499b423 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:22.440464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2d686-0833-4168-928b-e399a499b423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:22.472211 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-cda2d686-0833-4168-928b-e399a499b423 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:22.473430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cda2d686-0833-4168-928b-e399a499b423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:22.533488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:22.554654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd0471b-21e4-42a7-9baa-0ad183c7e772 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:22.593229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 362/724] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:57:22.621339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932187c-df32-497d-9e14-98240ee47eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:22.622031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932187c-df32-497d-9e14-98240ee47eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:22.622311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932187c-df32-497d-9e14-98240ee47eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:22.622581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932187c-df32-497d-9e14-98240ee47eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:22.623244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932187c-df32-497d-9e14-98240ee47eb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:22.623546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932187c-df32-497d-9e14-98240ee47eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:22.624912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932187c-df32-497d-9e14-98240ee47eb9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:22.637918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 363/725] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:22.643189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2d686-0833-4168-928b-e399a499b423 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:22.783803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2d686-0833-4168-928b-e399a499b423 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:22.842333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc979a3-e155-41b8-842d-1958d6bee219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:22.842972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc979a3-e155-41b8-842d-1958d6bee219 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:22.843307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc979a3-e155-41b8-842d-1958d6bee219 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:22.843521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc979a3-e155-41b8-842d-1958d6bee219 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:22.854696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc979a3-e155-41b8-842d-1958d6bee219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:22.856145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc979a3-e155-41b8-842d-1958d6bee219 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-dbc979a3-e155-41b8-842d-1958d6bee219 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:22.875282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:22.876229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:22.877145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:22.877145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:22.877145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:22.877145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 363/726] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:22.881932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932187c-df32-497d-9e14-98240ee47eb9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:22.950122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:23.012949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932187c-df32-497d-9e14-98240ee47eb9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:23.076835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dcaeb-dbed-494e-b62d-fb4f47795b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:23.077493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dcaeb-dbed-494e-b62d-fb4f47795b8d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:23.077768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dcaeb-dbed-494e-b62d-fb4f47795b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:23.078033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4dcaeb-dbed-494e-b62d-fb4f47795b8d None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:23.078677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4dcaeb-dbed-494e-b62d-fb4f47795b8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:23.078976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4dcaeb-dbed-494e-b62d-fb4f47795b8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:23.080326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4dcaeb-dbed-494e-b62d-fb4f47795b8d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:23.097931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 364/727] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:23.106179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc979a3-e155-41b8-842d-1958d6bee219 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:23.138049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:23.138975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dcaeb-dbed-494e-b62d-fb4f47795b8d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:23.252841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4dcaeb-dbed-494e-b62d-fb4f47795b8d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:23.268060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:23.299127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc979a3-e155-41b8-842d-1958d6bee219 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:23.337871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bae1a-97c8-4619-89d9-6a9d0d242a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:23.338670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bae1a-97c8-4619-89d9-6a9d0d242a77 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:23.339042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bae1a-97c8-4619-89d9-6a9d0d242a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:23.339325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843bae1a-97c8-4619-89d9-6a9d0d242a77 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:23.342466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43a10e-5709-430c-916d-0755b9374b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:23.343130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43a10e-5709-430c-916d-0755b9374b13 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:23.343411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43a10e-5709-430c-916d-0755b9374b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:23.343687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43a10e-5709-430c-916d-0755b9374b13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:23.348632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bae1a-97c8-4619-89d9-6a9d0d242a77 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:23.350055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843bae1a-97c8-4619-89d9-6a9d0d242a77 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-843bae1a-97c8-4619-89d9-6a9d0d242a77 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:23.364661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:23.365779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:23.366687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:23.366687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:23.366687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:23.366687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:23.366687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:23.366687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:23.366687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:23.366687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:23.366687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:23.366687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:23.369771 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 364/728] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 22 21:57:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:23.371968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bae1a-97c8-4619-89d9-6a9d0d242a77 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:23.389743 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43a10e-5709-430c-916d-0755b9374b13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2b43a10e-5709-430c-916d-0755b9374b13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:23.394636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:23.395541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:23.396364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 365/729] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:23.403214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43a10e-5709-430c-916d-0755b9374b13 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:23.469461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:23.499163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843bae1a-97c8-4619-89d9-6a9d0d242a77 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:23.545055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f04fd-cf21-423b-96b3-30d4a1a1560c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:23.545931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f04fd-cf21-423b-96b3-30d4a1a1560c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:23.546208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f04fd-cf21-423b-96b3-30d4a1a1560c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:23.546882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f04fd-cf21-423b-96b3-30d4a1a1560c None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:23.547885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f04fd-cf21-423b-96b3-30d4a1a1560c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:23.548190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f04fd-cf21-423b-96b3-30d4a1a1560c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:23.549258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43a10e-5709-430c-916d-0755b9374b13 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:23.549653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f04fd-cf21-423b-96b3-30d4a1a1560c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:23.588608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041b797-abbf-4787-808a-7d0bbdd5869b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:23.589335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041b797-abbf-4787-808a-7d0bbdd5869b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:23.589605 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041b797-abbf-4787-808a-7d0bbdd5869b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:23.589876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041b797-abbf-4787-808a-7d0bbdd5869b None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:23.590860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041b797-abbf-4787-808a-7d0bbdd5869b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:23.591166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041b797-abbf-4787-808a-7d0bbdd5869b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:23.592578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041b797-abbf-4787-808a-7d0bbdd5869b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:23.602804 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 366/730] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:57:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:23.604683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090e979-2658-49fd-b7f9-58a90a3b988b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:23.604967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090e979-2658-49fd-b7f9-58a90a3b988b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:23.605333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090e979-2658-49fd-b7f9-58a90a3b988b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:23.606811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 367/731] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:23.612427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3527807-88d5-48c0-8945-6533ad91098b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:23.612730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3527807-88d5-48c0-8945-6533ad91098b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:23.612970 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3527807-88d5-48c0-8945-6533ad91098b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:23.633982 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d3527807-88d5-48c0-8945-6533ad91098b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:23.635257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d3527807-88d5-48c0-8945-6533ad91098b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:23.672325 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:23.714832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 365/732] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 22 21:57:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:23.715995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f04fd-cf21-423b-96b3-30d4a1a1560c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:23.722786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 368/733] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 22 21:57:23.723841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3527807-88d5-48c0-8945-6533ad91098b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:23.836599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3527807-88d5-48c0-8945-6533ad91098b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:23.877724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c060-afbb-445c-8ed5-7c7c2633a049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:23.878613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c060-afbb-445c-8ed5-7c7c2633a049 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:23.879051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c060-afbb-445c-8ed5-7c7c2633a049 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:23.879474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c060-afbb-445c-8ed5-7c7c2633a049 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:23.892203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f04fd-cf21-423b-96b3-30d4a1a1560c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:23.919273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:23.971520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c3341-11aa-4a77-ab59-ba60d7338aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:23.972436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c3341-11aa-4a77-ab59-ba60d7338aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:23.972875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c3341-11aa-4a77-ab59-ba60d7338aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:23.974763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c3341-11aa-4a77-ab59-ba60d7338aa5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:23.981023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:23.997629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c3341-11aa-4a77-ab59-ba60d7338aa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:23.998188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c060-afbb-445c-8ed5-7c7c2633a049 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:24.000081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c3341-11aa-4a77-ab59-ba60d7338aa5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:24.001485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c060-afbb-445c-8ed5-7c7c2633a049 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e97c3341-11aa-4a77-ab59-ba60d7338aa5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:24.031133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:24.032287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:24.033355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:24.033355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:24.033355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:24.045175 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 366/734] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:24.047448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c3341-11aa-4a77-ab59-ba60d7338aa5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:24.187714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 369/735] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 22 21:57:24.188644 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c060-afbb-445c-8ed5-7c7c2633a049 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:24.220810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c3341-11aa-4a77-ab59-ba60d7338aa5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:24.252486 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:24.259624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50bcf1e-671f-4c06-8135-fffe1c99144f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:24.260215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bcf1e-671f-4c06-8135-fffe1c99144f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:24.260489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bcf1e-671f-4c06-8135-fffe1c99144f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:24.260756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50bcf1e-671f-4c06-8135-fffe1c99144f None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:24.261421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50bcf1e-671f-4c06-8135-fffe1c99144f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:24.261718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50bcf1e-671f-4c06-8135-fffe1c99144f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:24.263083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50bcf1e-671f-4c06-8135-fffe1c99144f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:24.274071 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 367/736] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:24.276168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c5f50-fac6-44af-9e68-899791355384 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:24.276454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c5f50-fac6-44af-9e68-899791355384 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:24.276729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c5f50-fac6-44af-9e68-899791355384 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:24.278151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 368/737] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:24.278676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c5f50-fac6-44af-9e68-899791355384 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:24.291509 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c060-afbb-445c-8ed5-7c7c2633a049 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:24.325674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1d9070-3eb6-468e-872c-3430efa5172b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:24.326286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d9070-3eb6-468e-872c-3430efa5172b None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:24.326557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d9070-3eb6-468e-872c-3430efa5172b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:24.326826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1d9070-3eb6-468e-872c-3430efa5172b None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:24.327462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d9070-3eb6-468e-872c-3430efa5172b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:24.328151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:24.328972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1d9070-3eb6-468e-872c-3430efa5172b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:24.344386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 370/738] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:57:24 2022] POST /identity/v3/services => generated 251 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:57:24.345619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87249a48-accc-4d41-b309-8bf55676ceb8 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:24.345903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87249a48-accc-4d41-b309-8bf55676ceb8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:24.346173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87249a48-accc-4d41-b309-8bf55676ceb8 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:24.347395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 371/739] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:24.348407 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81479891-a358-42dd-a17d-98ce64749a87 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:24.348693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81479891-a358-42dd-a17d-98ce64749a87 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:24.348963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81479891-a358-42dd-a17d-98ce64749a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:24.368926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-81479891-a358-42dd-a17d-98ce64749a87 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:24.370213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-81479891-a358-42dd-a17d-98ce64749a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:24.409119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c5f50-fac6-44af-9e68-899791355384 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:24.445910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ed618-6024-4823-a997-bd7edaad0256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:24.446557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed618-6024-4823-a997-bd7edaad0256 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:24.446824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed618-6024-4823-a997-bd7edaad0256 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:24.447127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed618-6024-4823-a997-bd7edaad0256 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:24.448775 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 372/740] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:24.449481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81479891-a358-42dd-a17d-98ce64749a87 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:24.470764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ed618-6024-4823-a997-bd7edaad0256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:24.472702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ed618-6024-4823-a997-bd7edaad0256 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-354ed618-6024-4823-a997-bd7edaad0256 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:24.495301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:24.496305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:24.497210 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:24.497210 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:24.497210 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:24.498787 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 369/741] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:24.500240 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7dfb-3725-41d0-902f-5ca1ef67a3df None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:24.500368 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7dfb-3725-41d0-902f-5ca1ef67a3df None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:24.500574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7dfb-3725-41d0-902f-5ca1ef67a3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:24.516389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:24.527109 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-858a7dfb-3725-41d0-902f-5ca1ef67a3df None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:24.528305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-858a7dfb-3725-41d0-902f-5ca1ef67a3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:24.568368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81479891-a358-42dd-a17d-98ce64749a87 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:24.614582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 370/742] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 22 21:57:24.615443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a7dfb-3725-41d0-902f-5ca1ef67a3df None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:24.617804 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db504ae-df75-4e9d-8fa4-6bc9dac7c7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:24.619031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db504ae-df75-4e9d-8fa4-6bc9dac7c7ea None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:24.619559 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db504ae-df75-4e9d-8fa4-6bc9dac7c7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:24.620030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db504ae-df75-4e9d-8fa4-6bc9dac7c7ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:24.629191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db504ae-df75-4e9d-8fa4-6bc9dac7c7ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:24.630902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db504ae-df75-4e9d-8fa4-6bc9dac7c7ea None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9db504ae-df75-4e9d-8fa4-6bc9dac7c7ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:24.644593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:24.645608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:24.646563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:24.646563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:24.646563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:24.646563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:24.646563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:24.646563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:24.646563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:24.646563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:24.646563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:24.646563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:24.649383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 373/743] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:24.657763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db504ae-df75-4e9d-8fa4-6bc9dac7c7ea None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:24.681722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:24.784212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a7dfb-3725-41d0-902f-5ca1ef67a3df None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:24.805825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db504ae-df75-4e9d-8fa4-6bc9dac7c7ea None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:24.826682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8a97d-9d89-40e1-bc9e-96d10c19c9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:24.827281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8a97d-9d89-40e1-bc9e-96d10c19c9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:24.827558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8a97d-9d89-40e1-bc9e-96d10c19c9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:24.827824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8a97d-9d89-40e1-bc9e-96d10c19c9c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:24.828454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8a97d-9d89-40e1-bc9e-96d10c19c9c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:24.828755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8a97d-9d89-40e1-bc9e-96d10c19c9c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:24.829445 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:24.830140 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8a97d-9d89-40e1-bc9e-96d10c19c9c2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:24.846950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 371/744] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:24.847820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8a97d-9d89-40e1-bc9e-96d10c19c9c2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:24.850486 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b9e27-4bbc-4dfb-84ce-63a59d02f213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:24.851392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b9e27-4bbc-4dfb-84ce-63a59d02f213 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:24.851982 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b9e27-4bbc-4dfb-84ce-63a59d02f213 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:24.852475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b9e27-4bbc-4dfb-84ce-63a59d02f213 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:24.853945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b9e27-4bbc-4dfb-84ce-63a59d02f213 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:24.854465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b9e27-4bbc-4dfb-84ce-63a59d02f213 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:24.856141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b9e27-4bbc-4dfb-84ce-63a59d02f213 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:24.879620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 374/745] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 22 21:57:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:24.880814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b9e27-4bbc-4dfb-84ce-63a59d02f213 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:24.942980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:24.963013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8a97d-9d89-40e1-bc9e-96d10c19c9c2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:24.985139 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b9e27-4bbc-4dfb-84ce-63a59d02f213 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:25.004807 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00babb8a-f38b-4e33-843a-388b393b6ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:25.005445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00babb8a-f38b-4e33-843a-388b393b6ecb None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:25.005720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00babb8a-f38b-4e33-843a-388b393b6ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:25.005990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00babb8a-f38b-4e33-843a-388b393b6ecb None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:25.006635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00babb8a-f38b-4e33-843a-388b393b6ecb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:25.007986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00babb8a-f38b-4e33-843a-388b393b6ecb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:25.023566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 372/746] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:57:24 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:57:25.024365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00babb8a-f38b-4e33-843a-388b393b6ecb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:25.046110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1a970-cdd3-48f4-8007-d6fd440cb2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:25.046903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1a970-cdd3-48f4-8007-d6fd440cb2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:25.047548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1a970-cdd3-48f4-8007-d6fd440cb2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:25.047974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1a970-cdd3-48f4-8007-d6fd440cb2e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:25.069608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1a970-cdd3-48f4-8007-d6fd440cb2e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:25.071301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1a970-cdd3-48f4-8007-d6fd440cb2e8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c9b1a970-cdd3-48f4-8007-d6fd440cb2e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:25.084391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:25.085443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:25.086356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:25.086356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:25.086356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:25.086356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:25.086356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:25.086356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:25.086356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:25.086356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:25.089298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 375/747] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:25.090874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1a970-cdd3-48f4-8007-d6fd440cb2e8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:25.136295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:25.158387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00babb8a-f38b-4e33-843a-388b393b6ecb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:25.215627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84cacf-af6c-4427-b5e1-b945043938a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:25.216289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84cacf-af6c-4427-b5e1-b945043938a1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:25.216561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84cacf-af6c-4427-b5e1-b945043938a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:25.216826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b84cacf-af6c-4427-b5e1-b945043938a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:25.236105 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:25.237664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1a970-cdd3-48f4-8007-d6fd440cb2e8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:25.278616 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41aa6d6-d697-4a59-95cf-4cc1506eee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:25.279247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41aa6d6-d697-4a59-95cf-4cc1506eee1a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:25.279522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41aa6d6-d697-4a59-95cf-4cc1506eee1a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:25.279853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41aa6d6-d697-4a59-95cf-4cc1506eee1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:25.291976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41aa6d6-d697-4a59-95cf-4cc1506eee1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b41aa6d6-d697-4a59-95cf-4cc1506eee1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:25.298013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:25.298989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 376/748] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:25.299886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41aa6d6-d697-4a59-95cf-4cc1506eee1a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:25.314665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84cacf-af6c-4427-b5e1-b945043938a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:25.317150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b84cacf-af6c-4427-b5e1-b945043938a1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:25.411917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:25.426136 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41aa6d6-d697-4a59-95cf-4cc1506eee1a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:25.454621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 373/749] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:57:25.455757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84cacf-af6c-4427-b5e1-b945043938a1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:25.467161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1160e-8270-45f1-bc2c-9dba29e74fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:25.467768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1160e-8270-45f1-bc2c-9dba29e74fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:25.468041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1160e-8270-45f1-bc2c-9dba29e74fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:25.468341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1160e-8270-45f1-bc2c-9dba29e74fe6 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:25.469221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a1160e-8270-45f1-bc2c-9dba29e74fe6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:25.469523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a1160e-8270-45f1-bc2c-9dba29e74fe6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:25.470930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a1160e-8270-45f1-bc2c-9dba29e74fe6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:25.490897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 377/750] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:25.490897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1160e-8270-45f1-bc2c-9dba29e74fe6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:25.511929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:25.581886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b84cacf-af6c-4427-b5e1-b945043938a1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:25.615558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1160e-8270-45f1-bc2c-9dba29e74fe6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:25.622736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b63bd-e25f-4727-9c63-8aaf4634bc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:25.623325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b63bd-e25f-4727-9c63-8aaf4634bc83 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:25.623699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b63bd-e25f-4727-9c63-8aaf4634bc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:25.624071 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b63bd-e25f-4727-9c63-8aaf4634bc83 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:25.624825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b63bd-e25f-4727-9c63-8aaf4634bc83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:25.625263 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b63bd-e25f-4727-9c63-8aaf4634bc83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:25.626713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b63bd-e25f-4727-9c63-8aaf4634bc83 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:25.636345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 374/751] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:57:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:25.637379 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b63bd-e25f-4727-9c63-8aaf4634bc83 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:25.665503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861277c7-e169-4573-a56d-80898b1be6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:25.667094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861277c7-e169-4573-a56d-80898b1be6ed None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:25.667937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861277c7-e169-4573-a56d-80898b1be6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:25.668357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861277c7-e169-4573-a56d-80898b1be6ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:25.682708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861277c7-e169-4573-a56d-80898b1be6ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:25.685044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861277c7-e169-4573-a56d-80898b1be6ed None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:25.698707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-861277c7-e169-4573-a56d-80898b1be6ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:25.708846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:25.709929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:25.710827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:25.710827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:25.710827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:25.718386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 378/752] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:25.719434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861277c7-e169-4573-a56d-80898b1be6ed None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:25.752619 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b63bd-e25f-4727-9c63-8aaf4634bc83 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:25.764542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:25.797943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ff2b4-85d2-4bc5-bc26-9b352977d4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:25.798689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ff2b4-85d2-4bc5-bc26-9b352977d4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:25.799089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ff2b4-85d2-4bc5-bc26-9b352977d4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:25.800286 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ff2b4-85d2-4bc5-bc26-9b352977d4c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:25.801140 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ff2b4-85d2-4bc5-bc26-9b352977d4c2 None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=bf3dde2bfc4240a8af6425778c5eddaf) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:25.801561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ff2b4-85d2-4bc5-bc26-9b352977d4c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:25.803130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736ff2b4-85d2-4bc5-bc26-9b352977d4c2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:25.810281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 375/753] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:25 2022] GET /identity/v3/role_assignments?user.id=bf3dde2bfc4240a8af6425778c5eddaf&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:25.812054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ff2b4-85d2-4bc5-bc26-9b352977d4c2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:25.862758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861277c7-e169-4573-a56d-80898b1be6ed None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:25.969408 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736ff2b4-85d2-4bc5-bc26-9b352977d4c2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:25.969867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338ad02-6031-47ff-aabe-ce95082fc409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:25.970531 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338ad02-6031-47ff-aabe-ce95082fc409 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:25.970808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338ad02-6031-47ff-aabe-ce95082fc409 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:25.971077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338ad02-6031-47ff-aabe-ce95082fc409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:25.989246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338ad02-6031-47ff-aabe-ce95082fc409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:25.990718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338ad02-6031-47ff-aabe-ce95082fc409 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1338ad02-6031-47ff-aabe-ce95082fc409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:26.015648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:26.016833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:26.017927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:26.017927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:26.017927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:26.023640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 379/754] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:26.025411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338ad02-6031-47ff-aabe-ce95082fc409 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:26.036747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39568628-72ed-408a-9831-8e04c2d952e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:26.038846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39568628-72ed-408a-9831-8e04c2d952e8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:26.039129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39568628-72ed-408a-9831-8e04c2d952e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:26.039400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39568628-72ed-408a-9831-8e04c2d952e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:26.040048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39568628-72ed-408a-9831-8e04c2d952e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:26.040714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39568628-72ed-408a-9831-8e04c2d952e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:26.042722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39568628-72ed-408a-9831-8e04c2d952e8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:26.065327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 376/755] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:26.073153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39568628-72ed-408a-9831-8e04c2d952e8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:26.119522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:26.133999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:26.202011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338ad02-6031-47ff-aabe-ce95082fc409 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:26.261131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39568628-72ed-408a-9831-8e04c2d952e8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:26.279196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74062aa3-aab1-4fc3-9294-99e80cbe1543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:26.279812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062aa3-aab1-4fc3-9294-99e80cbe1543 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:26.280086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062aa3-aab1-4fc3-9294-99e80cbe1543 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:26.280354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74062aa3-aab1-4fc3-9294-99e80cbe1543 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:26.280993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74062aa3-aab1-4fc3-9294-99e80cbe1543 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:26.281334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74062aa3-aab1-4fc3-9294-99e80cbe1543 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:26.282691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74062aa3-aab1-4fc3-9294-99e80cbe1543 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:26.302305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 380/756] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:26.305052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b151a-0e8e-4133-8f51-33bf22468e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:26.306462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b151a-0e8e-4133-8f51-33bf22468e88 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:26.307495 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74062aa3-aab1-4fc3-9294-99e80cbe1543 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:26.308378 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b151a-0e8e-4133-8f51-33bf22468e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:26.308830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b151a-0e8e-4133-8f51-33bf22468e88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:26.324359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b151a-0e8e-4133-8f51-33bf22468e88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:26.326078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b151a-0e8e-4133-8f51-33bf22468e88 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-610b151a-0e8e-4133-8f51-33bf22468e88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:26.344663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:26.345908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:26.346800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:26.346800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:26.346800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:26.346800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:26.346800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:26.346800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:26.346800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:26.346800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:26.353276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 377/757] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:26.357414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b151a-0e8e-4133-8f51-33bf22468e88 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:26.498058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74062aa3-aab1-4fc3-9294-99e80cbe1543 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:26.525288 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:26.540548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b151a-0e8e-4133-8f51-33bf22468e88 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:26.548217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:26.577973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e273e3-deaa-49e2-a765-954de7bc3566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:26.578925 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e273e3-deaa-49e2-a765-954de7bc3566 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:26.579487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e273e3-deaa-49e2-a765-954de7bc3566 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:26.579765 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e273e3-deaa-49e2-a765-954de7bc3566 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:26.583558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dbbe9-7063-44ac-8207-b4d55b87966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:26.584157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dbbe9-7063-44ac-8207-b4d55b87966b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:26.584436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dbbe9-7063-44ac-8207-b4d55b87966b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:26.584708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7dbbe9-7063-44ac-8207-b4d55b87966b None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:26.585424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dbbe9-7063-44ac-8207-b4d55b87966b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:26.585728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dbbe9-7063-44ac-8207-b4d55b87966b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:26.587858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7dbbe9-7063-44ac-8207-b4d55b87966b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:26.594207 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e273e3-deaa-49e2-a765-954de7bc3566 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:26.595659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e273e3-deaa-49e2-a765-954de7bc3566 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:26.601534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 378/758] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:26.610097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dbbe9-7063-44ac-8207-b4d55b87966b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-39e273e3-deaa-49e2-a765-954de7bc3566 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:26.618065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:26.619522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:26.620455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:26.620455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:26.620455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:26.620455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:26.620455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:26.620455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:26.620455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:26.620455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:26.620455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:57:26.620455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:26.621603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 381/759] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 22 21:57:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:26.634679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e273e3-deaa-49e2-a765-954de7bc3566 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:26.745812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7dbbe9-7063-44ac-8207-b4d55b87966b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:26.771737 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e273e3-deaa-49e2-a765-954de7bc3566 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:26.786071 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae36de-2af3-4888-9429-f7a5ba2f1c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:26.786675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae36de-2af3-4888-9429-f7a5ba2f1c8f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:26.787066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae36de-2af3-4888-9429-f7a5ba2f1c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:26.787346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae36de-2af3-4888-9429-f7a5ba2f1c8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:26.788177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae36de-2af3-4888-9429-f7a5ba2f1c8f None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=471e7550ae7a4f909851215d3ed148c6) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:26.788492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae36de-2af3-4888-9429-f7a5ba2f1c8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:26.789954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae36de-2af3-4888-9429-f7a5ba2f1c8f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:26.796566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 379/760] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:26 2022] GET /identity/v3/role_assignments?user.id=471e7550ae7a4f909851215d3ed148c6&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:26.798084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b386e-23dd-4ab7-884d-c8668f34af80 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:26.798615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b386e-23dd-4ab7-884d-c8668f34af80 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:26.798615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b386e-23dd-4ab7-884d-c8668f34af80 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:26.800184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 380/761] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:26.813420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183afd-9990-4ec0-a31a-73361f90badc None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:26.813729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183afd-9990-4ec0-a31a-73361f90badc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:26.813972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59183afd-9990-4ec0-a31a-73361f90badc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:26.814759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908639c-dc23-425f-93f1-3c51b54b2502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:26.815374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908639c-dc23-425f-93f1-3c51b54b2502 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:26.815647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908639c-dc23-425f-93f1-3c51b54b2502 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:26.815914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908639c-dc23-425f-93f1-3c51b54b2502 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:26.816653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908639c-dc23-425f-93f1-3c51b54b2502 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:26.816956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908639c-dc23-425f-93f1-3c51b54b2502 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:26.818526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908639c-dc23-425f-93f1-3c51b54b2502 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:26.847243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 382/762] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 22 21:57:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:26.848627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908639c-dc23-425f-93f1-3c51b54b2502 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:26.858492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-59183afd-9990-4ec0-a31a-73361f90badc None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:26.859742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-59183afd-9990-4ec0-a31a-73361f90badc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:26.882310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:26.885946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:26.995955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908639c-dc23-425f-93f1-3c51b54b2502 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:27.022502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 381/763] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:57:27.064149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0b65d-772d-4fc1-a79e-9335e46ac7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:27.064149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0b65d-772d-4fc1-a79e-9335e46ac7bf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:27.064149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0b65d-772d-4fc1-a79e-9335e46ac7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:27.064149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e0b65d-772d-4fc1-a79e-9335e46ac7bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:27.076161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0b65d-772d-4fc1-a79e-9335e46ac7bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:27.077645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e0b65d-772d-4fc1-a79e-9335e46ac7bf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e3e0b65d-772d-4fc1-a79e-9335e46ac7bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:27.099201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:27.100187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:27.101088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:27.101088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:27.101088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:27.105207 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 383/764] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:27.110088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59183afd-9990-4ec0-a31a-73361f90badc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:27.257023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59183afd-9990-4ec0-a31a-73361f90badc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:27.297850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d12a0-b3a5-4427-9a92-765636c56805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:27.298378 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d12a0-b3a5-4427-9a92-765636c56805 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:27.298661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d12a0-b3a5-4427-9a92-765636c56805 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:27.298936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d12a0-b3a5-4427-9a92-765636c56805 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:27.299578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d12a0-b3a5-4427-9a92-765636c56805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:27.299943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d12a0-b3a5-4427-9a92-765636c56805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:27.301395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23d12a0-b3a5-4427-9a92-765636c56805 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:27.318442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 382/765] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:27.324518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0b65d-772d-4fc1-a79e-9335e46ac7bf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:27.437581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d12a0-b3a5-4427-9a92-765636c56805 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:27.444021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:27.451377 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:27.463682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e0b65d-772d-4fc1-a79e-9335e46ac7bf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:27.513855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50fccb-0d4c-472e-801f-ebbe9cfbee07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:27.514485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50fccb-0d4c-472e-801f-ebbe9cfbee07 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:27.514756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50fccb-0d4c-472e-801f-ebbe9cfbee07 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:27.515023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50fccb-0d4c-472e-801f-ebbe9cfbee07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:27.527328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50fccb-0d4c-472e-801f-ebbe9cfbee07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:27.528772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50fccb-0d4c-472e-801f-ebbe9cfbee07 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-da50fccb-0d4c-472e-801f-ebbe9cfbee07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:27.542350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:27.543328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:27.544248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:27.544248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:27.544248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:27.544248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:27.544248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:27.544248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:27.544248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:27.544248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:27.549035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 384/766] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:27.550169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50fccb-0d4c-472e-801f-ebbe9cfbee07 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:27.621693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d12a0-b3a5-4427-9a92-765636c56805 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:27.677538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc3402-ee31-4f9e-a6ce-cd08d20ffb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:27.678181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc3402-ee31-4f9e-a6ce-cd08d20ffb2a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:27.678460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc3402-ee31-4f9e-a6ce-cd08d20ffb2a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:27.678734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc3402-ee31-4f9e-a6ce-cd08d20ffb2a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:27.679323 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc3402-ee31-4f9e-a6ce-cd08d20ffb2a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:27.683075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc3402-ee31-4f9e-a6ce-cd08d20ffb2a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:27.692722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50fccb-0d4c-472e-801f-ebbe9cfbee07 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-2ddc3402-ee31-4f9e-a6ce-cd08d20ffb2a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:27.700746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:27.701828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:27.702960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 383/767] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 22 21:57:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:27.704120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2195-2371-4a61-a809-6aead8427165 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:27.704280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2195-2371-4a61-a809-6aead8427165 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:27.705326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e2195-2371-4a61-a809-6aead8427165 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:27.709031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 384/768] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:27.710221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e2195-2371-4a61-a809-6aead8427165 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:27.744204 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e199e-43a8-4073-87f8-0929a00750ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:27.745385 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e199e-43a8-4073-87f8-0929a00750ec None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:27.745961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e199e-43a8-4073-87f8-0929a00750ec None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:27.746425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e199e-43a8-4073-87f8-0929a00750ec None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:27.747248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e199e-43a8-4073-87f8-0929a00750ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:27.747709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e199e-43a8-4073-87f8-0929a00750ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:27.749301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e199e-43a8-4073-87f8-0929a00750ec None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:27.768192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 385/769] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:27.768192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f65c8-b069-49d8-953b-819dc211b985 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:27.768192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f65c8-b069-49d8-953b-819dc211b985 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:27.768192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f65c8-b069-49d8-953b-819dc211b985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:27.781436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:27.797778 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-530f65c8-b069-49d8-953b-819dc211b985 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:27.798872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-530f65c8-b069-49d8-953b-819dc211b985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:27.803352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:27.842497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e2195-2371-4a61-a809-6aead8427165 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:27.881195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 386/770] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:27.881195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f65c8-b069-49d8-953b-819dc211b985 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:27.885656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ea240-21bb-4aad-ac03-276c27195a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:27.886513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ea240-21bb-4aad-ac03-276c27195a69 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:27.887012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ea240-21bb-4aad-ac03-276c27195a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:27.887351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ea240-21bb-4aad-ac03-276c27195a69 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:27.888048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ea240-21bb-4aad-ac03-276c27195a69 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:27.888415 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ea240-21bb-4aad-ac03-276c27195a69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:27.889858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ea240-21bb-4aad-ac03-276c27195a69 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:27.902052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 385/771] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 22 21:57:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:27.907626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ea240-21bb-4aad-ac03-276c27195a69 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:28.061856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f65c8-b069-49d8-953b-819dc211b985 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:28.105824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ea240-21bb-4aad-ac03-276c27195a69 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:28.118009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:28.129128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f20d2-6718-4bd0-9f8c-a726c7b02059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:28.130046 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f20d2-6718-4bd0-9f8c-a726c7b02059 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:28.130502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f20d2-6718-4bd0-9f8c-a726c7b02059 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:28.130959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f20d2-6718-4bd0-9f8c-a726c7b02059 None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/d011e54e12674eaeb82c68da628ecb9d/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:28.133085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f20d2-6718-4bd0-9f8c-a726c7b02059 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=d011e54e12674eaeb82c68da628ecb9d, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:28.136164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348f20d2-6718-4bd0-9f8c-a726c7b02059 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:28.151335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 387/772] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:57:27 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/d011e54e12674eaeb82c68da628ecb9d/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:57:28.182911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373d405-07c8-4806-968a-7ed2a22c2e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:28.183836 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373d405-07c8-4806-968a-7ed2a22c2e90 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:28.184293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373d405-07c8-4806-968a-7ed2a22c2e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:28.184769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7373d405-07c8-4806-968a-7ed2a22c2e90 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:28.185892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373d405-07c8-4806-968a-7ed2a22c2e90 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:28.186376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373d405-07c8-4806-968a-7ed2a22c2e90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:28.188001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7373d405-07c8-4806-968a-7ed2a22c2e90 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:28.189800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:28.195512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 386/773] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 22 21:57:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:28.201867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f20d2-6718-4bd0-9f8c-a726c7b02059 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:28.274928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373d405-07c8-4806-968a-7ed2a22c2e90 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:28.396017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f20d2-6718-4bd0-9f8c-a726c7b02059 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:28.437984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7373d405-07c8-4806-968a-7ed2a22c2e90 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:28.449430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77e751-da15-40a5-8435-505602a5584c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:28.450051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77e751-da15-40a5-8435-505602a5584c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:28.450388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77e751-da15-40a5-8435-505602a5584c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:28.450670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c77e751-da15-40a5-8435-505602a5584c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:28.451369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77e751-da15-40a5-8435-505602a5584c None admin] RBAC: Adding query filter params (interface=public, service_id=0ccaed734475486992b236e900473c42, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:28.451677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77e751-da15-40a5-8435-505602a5584c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:28.453046 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c77e751-da15-40a5-8435-505602a5584c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:28.462289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 388/774] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 22 21:57:28 2022] GET /identity/v3/endpoints?service_id=0ccaed734475486992b236e900473c42&interface=public®ion_id=RegionOne => generated 194 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:28.468061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77e751-da15-40a5-8435-505602a5584c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:28.479926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537cfe0-bc7f-4454-8eb1-37bd99e78ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:28.480840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537cfe0-bc7f-4454-8eb1-37bd99e78ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:28.481321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537cfe0-bc7f-4454-8eb1-37bd99e78ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:28.481737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5537cfe0-bc7f-4454-8eb1-37bd99e78ef9 None admin] PATH_INFO: `/v3/services/network` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:28.482584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537cfe0-bc7f-4454-8eb1-37bd99e78ef9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:28.484070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5537cfe0-bc7f-4454-8eb1-37bd99e78ef9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-5537cfe0-bc7f-4454-8eb1-37bd99e78ef9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:28.494542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:28.495544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:28.496446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 387/775] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:28 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:28.503090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537cfe0-bc7f-4454-8eb1-37bd99e78ef9 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:28.544247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:28.573971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:28.585486 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c77e751-da15-40a5-8435-505602a5584c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:28.616978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5537cfe0-bc7f-4454-8eb1-37bd99e78ef9 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:28.622839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16646d13-25c6-4f3f-b873-a7b01af6bd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:28.623863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646d13-25c6-4f3f-b873-a7b01af6bd3f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:28.624398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646d13-25c6-4f3f-b873-a7b01af6bd3f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:28.624913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646d13-25c6-4f3f-b873-a7b01af6bd3f None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:28.626307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16646d13-25c6-4f3f-b873-a7b01af6bd3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:28.628490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16646d13-25c6-4f3f-b873-a7b01af6bd3f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:28.645241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 389/776] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 22 21:57:28 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:28.646921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ff38a-f0dd-447e-9fdd-92035c975f81 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:28.647260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ff38a-f0dd-447e-9fdd-92035c975f81 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:28.647628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06ff38a-f0dd-447e-9fdd-92035c975f81 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:28.650041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 390/777] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:28.658342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae61af1-34c9-4db5-896d-9e3d5209a724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:28.659238 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae61af1-34c9-4db5-896d-9e3d5209a724 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:28.659668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae61af1-34c9-4db5-896d-9e3d5209a724 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:28.660088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae61af1-34c9-4db5-896d-9e3d5209a724 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:28.660886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae61af1-34c9-4db5-896d-9e3d5209a724 None admin] RBAC: Adding query filter params (name=network) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:28.661380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae61af1-34c9-4db5-896d-9e3d5209a724 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:28.662935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae61af1-34c9-4db5-896d-9e3d5209a724 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:28.664931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec65e09-db92-46ea-83bb-e872e915d45f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:28.665647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec65e09-db92-46ea-83bb-e872e915d45f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:28.665961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec65e09-db92-46ea-83bb-e872e915d45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:28.669272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 388/778] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:57:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:28.682277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae61af1-34c9-4db5-896d-9e3d5209a724 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:28.715006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7ec65e09-db92-46ea-83bb-e872e915d45f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:28.716498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7ec65e09-db92-46ea-83bb-e872e915d45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:28.846682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae61af1-34c9-4db5-896d-9e3d5209a724 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:28.862001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 391/779] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:57:28.893545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e521316-4bae-402e-a352-9ba96ff8cb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:28.894528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e521316-4bae-402e-a352-9ba96ff8cb25 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:28.894964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e521316-4bae-402e-a352-9ba96ff8cb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:28.895383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e521316-4bae-402e-a352-9ba96ff8cb25 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:28.896158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e521316-4bae-402e-a352-9ba96ff8cb25 None admin] RBAC: Adding query filter params (type=network) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:28.896695 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e521316-4bae-402e-a352-9ba96ff8cb25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:28.898395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e521316-4bae-402e-a352-9ba96ff8cb25 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:28.901564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:28.906277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 389/780] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:57:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:28.915902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec65e09-db92-46ea-83bb-e872e915d45f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:28.972638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:29.026278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec65e09-db92-46ea-83bb-e872e915d45f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:29.060927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f68f4-613a-49d4-8df3-054e10c17c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:29.064293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f68f4-613a-49d4-8df3-054e10c17c8c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:29.064293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f68f4-613a-49d4-8df3-054e10c17c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:29.064293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966f68f4-613a-49d4-8df3-054e10c17c8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:29.064293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f68f4-613a-49d4-8df3-054e10c17c8c None admin] RBAC: Adding query filter params (interface=public, service_id=4b83a391ebbf4f6283c9b3fcd299d48e, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:29.064293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f68f4-613a-49d4-8df3-054e10c17c8c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:29.064671 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966f68f4-613a-49d4-8df3-054e10c17c8c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:29.077079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 392/781] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 22 21:57:28 2022] GET /identity/v3/endpoints?service_id=4b83a391ebbf4f6283c9b3fcd299d48e&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:29.082201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e521316-4bae-402e-a352-9ba96ff8cb25 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:29.199170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e521316-4bae-402e-a352-9ba96ff8cb25 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:29.237572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78189053-2361-41d7-bf39-9300a86ca88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:29.238571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78189053-2361-41d7-bf39-9300a86ca88c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:29.239087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78189053-2361-41d7-bf39-9300a86ca88c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:29.239591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78189053-2361-41d7-bf39-9300a86ca88c None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:29.240379 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78189053-2361-41d7-bf39-9300a86ca88c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:29.241948 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78189053-2361-41d7-bf39-9300a86ca88c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:29.253950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f68f4-613a-49d4-8df3-054e10c17c8c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:29.264588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 390/782] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 22 21:57:29 2022] GET /identity/v3/services => generated 772 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:29.351531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a36450-52e3-45d7-abc1-d9461abc24b8 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:29.352174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a36450-52e3-45d7-abc1-d9461abc24b8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:29.353209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a36450-52e3-45d7-abc1-d9461abc24b8 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:29.354819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 391/783] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:57:29.360431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87772c-0289-4f45-a4a9-d7e7b6ea2281 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:29.361011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87772c-0289-4f45-a4a9-d7e7b6ea2281 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:29.361544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f87772c-0289-4f45-a4a9-d7e7b6ea2281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:29.401887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5f87772c-0289-4f45-a4a9-d7e7b6ea2281 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:29.406134 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5f87772c-0289-4f45-a4a9-d7e7b6ea2281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:29.415951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966f68f4-613a-49d4-8df3-054e10c17c8c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:29.433545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:29.455599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905b28a-8820-4a6a-86fd-2ebee586c239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:29.456557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905b28a-8820-4a6a-86fd-2ebee586c239 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:29.457499 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905b28a-8820-4a6a-86fd-2ebee586c239 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:29.458002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905b28a-8820-4a6a-86fd-2ebee586c239 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:29.471620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905b28a-8820-4a6a-86fd-2ebee586c239 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6905b28a-8820-4a6a-86fd-2ebee586c239 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:29.476893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:29.478744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:29.479612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:29.482386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 393/784] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:29.487745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905b28a-8820-4a6a-86fd-2ebee586c239 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:29.522462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:29.544442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 392/785] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:57:29.724459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905b28a-8820-4a6a-86fd-2ebee586c239 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:29.764585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff806e0-7715-46d4-a812-018fd3b9a3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:29.765275 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff806e0-7715-46d4-a812-018fd3b9a3ef None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:29.765539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff806e0-7715-46d4-a812-018fd3b9a3ef None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:29.765806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff806e0-7715-46d4-a812-018fd3b9a3ef None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:29.766516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff806e0-7715-46d4-a812-018fd3b9a3ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:29.766812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff806e0-7715-46d4-a812-018fd3b9a3ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:29.768177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff806e0-7715-46d4-a812-018fd3b9a3ef None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:29.777551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 394/786] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:29.782735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87772c-0289-4f45-a4a9-d7e7b6ea2281 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:29.869980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff806e0-7715-46d4-a812-018fd3b9a3ef None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:29.906887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:29.927349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f87772c-0289-4f45-a4a9-d7e7b6ea2281 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:29.976995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff806e0-7715-46d4-a812-018fd3b9a3ef None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:30.010287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71160bf-9dbe-4fec-8582-e7cfbb2bdd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:30.011408 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71160bf-9dbe-4fec-8582-e7cfbb2bdd79 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:30.011909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71160bf-9dbe-4fec-8582-e7cfbb2bdd79 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:30.012397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71160bf-9dbe-4fec-8582-e7cfbb2bdd79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:30.015868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88993415-a746-4c68-80f0-6a7a455068c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:30.016829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88993415-a746-4c68-80f0-6a7a455068c8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:30.017382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88993415-a746-4c68-80f0-6a7a455068c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:30.017853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88993415-a746-4c68-80f0-6a7a455068c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:30.030002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88993415-a746-4c68-80f0-6a7a455068c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-88993415-a746-4c68-80f0-6a7a455068c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:30.035064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:30.035963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:30.036776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:30.039425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 395/787] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:30.046255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88993415-a746-4c68-80f0-6a7a455068c8 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:30.054250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71160bf-9dbe-4fec-8582-e7cfbb2bdd79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:30.057605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71160bf-9dbe-4fec-8582-e7cfbb2bdd79 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f71160bf-9dbe-4fec-8582-e7cfbb2bdd79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:30.098500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:30.099490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:30.100387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:30.100387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:30.100387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:30.100387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 393/788] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:30.122157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71160bf-9dbe-4fec-8582-e7cfbb2bdd79 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:30.164388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:30.171433 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88993415-a746-4c68-80f0-6a7a455068c8 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:30.208057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd0d79-da13-4489-ba4e-0c2fe258cf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:30.209017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd0d79-da13-4489-ba4e-0c2fe258cf3a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:30.209531 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd0d79-da13-4489-ba4e-0c2fe258cf3a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:30.209992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecd0d79-da13-4489-ba4e-0c2fe258cf3a None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:30.210964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd0d79-da13-4489-ba4e-0c2fe258cf3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:30.211765 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:30.212870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd0d79-da13-4489-ba4e-0c2fe258cf3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:30.214492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecd0d79-da13-4489-ba4e-0c2fe258cf3a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:30.223797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 396/789] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:57:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:30.229717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd0d79-da13-4489-ba4e-0c2fe258cf3a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:30.299239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71160bf-9dbe-4fec-8582-e7cfbb2bdd79 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:30.340665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecd0d79-da13-4489-ba4e-0c2fe258cf3a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:30.378409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48d526-93ad-4c60-aad6-db5d64973661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:30.379459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48d526-93ad-4c60-aad6-db5d64973661 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:30.379968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48d526-93ad-4c60-aad6-db5d64973661 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:30.380445 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae48d526-93ad-4c60-aad6-db5d64973661 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:30.392383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48d526-93ad-4c60-aad6-db5d64973661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:30.394293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae48d526-93ad-4c60-aad6-db5d64973661 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:30.405830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad96ce8-1b86-4daa-9f32-75dd689af060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:30.406450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad96ce8-1b86-4daa-9f32-75dd689af060 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:30.406732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad96ce8-1b86-4daa-9f32-75dd689af060 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:30.407008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad96ce8-1b86-4daa-9f32-75dd689af060 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:30.407655 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad96ce8-1b86-4daa-9f32-75dd689af060 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:30.407959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad96ce8-1b86-4daa-9f32-75dd689af060 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:30.409362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad96ce8-1b86-4daa-9f32-75dd689af060 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ae48d526-93ad-4c60-aad6-db5d64973661 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:30.412064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:30.412988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:30.413949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:30.413949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:30.413949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:30.416571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 397/790] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:30.421708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48d526-93ad-4c60-aad6-db5d64973661 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:30.431058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 394/791] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:30.437508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:30.438056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad96ce8-1b86-4daa-9f32-75dd689af060 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:30.536817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae48d526-93ad-4c60-aad6-db5d64973661 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:30.598128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1634d4-6429-44aa-876b-d0bb421ed54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:30.599684 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1634d4-6429-44aa-876b-d0bb421ed54c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:30.600334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1634d4-6429-44aa-876b-d0bb421ed54c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:30.603767 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1634d4-6429-44aa-876b-d0bb421ed54c None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:30.604731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1634d4-6429-44aa-876b-d0bb421ed54c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:30.605285 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1634d4-6429-44aa-876b-d0bb421ed54c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:30.606864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1634d4-6429-44aa-876b-d0bb421ed54c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:30.630821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 398/792] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:30.636256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1634d4-6429-44aa-876b-d0bb421ed54c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:30.664647 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:30.675228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad96ce8-1b86-4daa-9f32-75dd689af060 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:30.719028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:30.757378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1634d4-6429-44aa-876b-d0bb421ed54c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:30.765893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5693bc-3ffc-409c-afb7-6f6dc3fa0635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:30.766805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5693bc-3ffc-409c-afb7-6f6dc3fa0635 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:30.767216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5693bc-3ffc-409c-afb7-6f6dc3fa0635 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:30.767603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5693bc-3ffc-409c-afb7-6f6dc3fa0635 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:30.792070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5693bc-3ffc-409c-afb7-6f6dc3fa0635 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:30.794415 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5693bc-3ffc-409c-afb7-6f6dc3fa0635 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:30.802641 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba8ae1-c39a-4b76-9613-10c26e59b0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:30.806696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba8ae1-c39a-4b76-9613-10c26e59b0c5 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:30.807222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba8ae1-c39a-4b76-9613-10c26e59b0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:30.807696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba8ae1-c39a-4b76-9613-10c26e59b0c5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:30.818387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba8ae1-c39a-4b76-9613-10c26e59b0c5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:30.820071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aba8ae1-c39a-4b76-9613-10c26e59b0c5 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-8f5693bc-3ffc-409c-afb7-6f6dc3fa0635 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:30.826629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:30.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:30.828590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:30.828590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:30.828590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:30.828590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:30.828590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:30.828590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:30.828590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:30.828590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:30.828590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:30.828590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:30.828590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 395/793] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 22 21:57:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:30.833581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5693bc-3ffc-409c-afb7-6f6dc3fa0635 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6aba8ae1-c39a-4b76-9613-10c26e59b0c5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:30.835783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:30.836797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:30.837989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:30.837989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:30.837989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:30.837989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:30.837989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:30.837989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:30.837989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:30.837989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:30.837989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:30.837989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:30.840898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 399/794] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:30.847720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba8ae1-c39a-4b76-9613-10c26e59b0c5 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:30.966164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5693bc-3ffc-409c-afb7-6f6dc3fa0635 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:30.994894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba8ae1-c39a-4b76-9613-10c26e59b0c5 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:31.003809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0f3e35-385b-42d0-88fb-63e3184e6f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:31.004493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f3e35-385b-42d0-88fb-63e3184e6f44 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.004841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f3e35-385b-42d0-88fb-63e3184e6f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.005216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f3e35-385b-42d0-88fb-63e3184e6f44 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.006008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f3e35-385b-42d0-88fb-63e3184e6f44 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:31.006382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f3e35-385b-42d0-88fb-63e3184e6f44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:31.007899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0f3e35-385b-42d0-88fb-63e3184e6f44 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:31.028304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 396/795] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 22 21:57:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:31.031065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:31.034392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66069a68-5861-420b-bf2f-9c986f4d9558 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.034789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66069a68-5861-420b-bf2f-9c986f4d9558 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.035127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66069a68-5861-420b-bf2f-9c986f4d9558 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.036805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 397/796] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:31.037648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66069a68-5861-420b-bf2f-9c986f4d9558 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:31.041619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e672ad22-ca21-4e7f-b1b0-6a26e13ee935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:31.042716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672ad22-ca21-4e7f-b1b0-6a26e13ee935 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.043130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672ad22-ca21-4e7f-b1b0-6a26e13ee935 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.043659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e672ad22-ca21-4e7f-b1b0-6a26e13ee935 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.044767 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672ad22-ca21-4e7f-b1b0-6a26e13ee935 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:31.045160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672ad22-ca21-4e7f-b1b0-6a26e13ee935 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:31.047673 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e672ad22-ca21-4e7f-b1b0-6a26e13ee935 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:31.059749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:31.081816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 400/797] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 22 21:57:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:31.083965 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c7deb-692c-4fa4-b7ef-d2dcaa2bed0e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.084573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c7deb-692c-4fa4-b7ef-d2dcaa2bed0e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.085207 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c7deb-692c-4fa4-b7ef-d2dcaa2bed0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.116006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ef0c7deb-692c-4fa4-b7ef-d2dcaa2bed0e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:31.117807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ef0c7deb-692c-4fa4-b7ef-d2dcaa2bed0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:31.173214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66069a68-5861-420b-bf2f-9c986f4d9558 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:31.212974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 401/798] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:31.212974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c7deb-692c-4fa4-b7ef-d2dcaa2bed0e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:31.215907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03168978-78bb-4a27-a9f0-7b06bd183e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:31.216585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168978-78bb-4a27-a9f0-7b06bd183e8a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.216863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168978-78bb-4a27-a9f0-7b06bd183e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.217151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03168978-78bb-4a27-a9f0-7b06bd183e8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.236146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03168978-78bb-4a27-a9f0-7b06bd183e8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:31.237627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03168978-78bb-4a27-a9f0-7b06bd183e8a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-03168978-78bb-4a27-a9f0-7b06bd183e8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:31.266330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:31.267289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:31.268601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:31.268601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:31.268601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:31.268601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 398/799] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:31.275903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03168978-78bb-4a27-a9f0-7b06bd183e8a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:31.396172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:31.413998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c7deb-692c-4fa4-b7ef-d2dcaa2bed0e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:31.453471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:31.463841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03168978-78bb-4a27-a9f0-7b06bd183e8a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:31.479683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c33ce88-56d1-47d0-a595-b9c31bb830aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:31.480328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33ce88-56d1-47d0-a595-b9c31bb830aa None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.480605 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33ce88-56d1-47d0-a595-b9c31bb830aa None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.480876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c33ce88-56d1-47d0-a595-b9c31bb830aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.495348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33ce88-56d1-47d0-a595-b9c31bb830aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:31.497224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c33ce88-56d1-47d0-a595-b9c31bb830aa None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:31.502487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec23aa9-5141-41ec-9b59-4f82ff0afb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:31.503093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec23aa9-5141-41ec-9b59-4f82ff0afb9a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.503374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec23aa9-5141-41ec-9b59-4f82ff0afb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.503646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec23aa9-5141-41ec-9b59-4f82ff0afb9a None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.504289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec23aa9-5141-41ec-9b59-4f82ff0afb9a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:31.504653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec23aa9-5141-41ec-9b59-4f82ff0afb9a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:31.506055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec23aa9-5141-41ec-9b59-4f82ff0afb9a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6c33ce88-56d1-47d0-a595-b9c31bb830aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:31.514845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:31.515784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:31.517202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:31.517202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:31.517202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:31.517202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 402/800] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:31.517556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf871d4-1b4d-45fd-a953-ff32788941ed None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.519514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf871d4-1b4d-45fd-a953-ff32788941ed None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.519514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf871d4-1b4d-45fd-a953-ff32788941ed None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.519676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 403/801] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:31.521800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf871d4-1b4d-45fd-a953-ff32788941ed None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:31.527767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 399/802] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:31.529091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efadb0-0a1e-49d7-8ea1-1ddfbb934ed0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.529413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efadb0-0a1e-49d7-8ea1-1ddfbb934ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.529684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1efadb0-0a1e-49d7-8ea1-1ddfbb934ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.549688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c1efadb0-0a1e-49d7-8ea1-1ddfbb934ed0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:31.551052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c1efadb0-0a1e-49d7-8ea1-1ddfbb934ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:31.622543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 400/803] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:31.623395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efadb0-0a1e-49d7-8ea1-1ddfbb934ed0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:31.644821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf871d4-1b4d-45fd-a953-ff32788941ed None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:31.685242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:31.694817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27921c72-b457-4602-91df-4ad39e1cf2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:31.695951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27921c72-b457-4602-91df-4ad39e1cf2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.697152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27921c72-b457-4602-91df-4ad39e1cf2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.697626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27921c72-b457-4602-91df-4ad39e1cf2b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.698711 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27921c72-b457-4602-91df-4ad39e1cf2b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:31.699186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27921c72-b457-4602-91df-4ad39e1cf2b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:31.701905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27921c72-b457-4602-91df-4ad39e1cf2b2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:31.718367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 404/804] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:31.720044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27921c72-b457-4602-91df-4ad39e1cf2b2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:31.733514 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1efadb0-0a1e-49d7-8ea1-1ddfbb934ed0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:31.760059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:31.770241 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e12fd-efbf-4e02-bfbb-e5fa5fa6d764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:31.770953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e12fd-efbf-4e02-bfbb-e5fa5fa6d764 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.771304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e12fd-efbf-4e02-bfbb-e5fa5fa6d764 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.771665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e12fd-efbf-4e02-bfbb-e5fa5fa6d764 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.780470 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e12fd-efbf-4e02-bfbb-e5fa5fa6d764 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:31.782147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e12fd-efbf-4e02-bfbb-e5fa5fa6d764 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-dd8e12fd-efbf-4e02-bfbb-e5fa5fa6d764 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:31.795421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:31.796324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:31.797538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:31.797538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:31.797538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:31.797538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:31.797538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:31.797538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:31.797538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:31.797538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:31.797538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 401/805] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:31.798746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e12fd-efbf-4e02-bfbb-e5fa5fa6d764 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:31.847239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27921c72-b457-4602-91df-4ad39e1cf2b2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:31.896236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58714944-7f77-4ad8-82b4-8be28f0e4ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:31.897549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58714944-7f77-4ad8-82b4-8be28f0e4ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.897932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58714944-7f77-4ad8-82b4-8be28f0e4ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.898569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58714944-7f77-4ad8-82b4-8be28f0e4ea3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.912876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58714944-7f77-4ad8-82b4-8be28f0e4ea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:31.916733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e12fd-efbf-4e02-bfbb-e5fa5fa6d764 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-58714944-7f77-4ad8-82b4-8be28f0e4ea3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:31.920016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:31.921085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:31.921976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:31.923648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 405/806] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:31.923648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58714944-7f77-4ad8-82b4-8be28f0e4ea3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:31.939054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:31.959218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edca83d-9a61-40d5-9b34-164136eae738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:31.959840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edca83d-9a61-40d5-9b34-164136eae738 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:31.960118 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edca83d-9a61-40d5-9b34-164136eae738 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:31.960397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edca83d-9a61-40d5-9b34-164136eae738 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:31.969548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edca83d-9a61-40d5-9b34-164136eae738 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:31.971113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edca83d-9a61-40d5-9b34-164136eae738 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-2edca83d-9a61-40d5-9b34-164136eae738 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:31.983612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:31.984509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:31.985452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:31.985452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:31.985452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:31.985452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:31.985452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:31.985452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:31.985452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:31.985452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:31.985452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 402/807] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:31.986656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edca83d-9a61-40d5-9b34-164136eae738 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:32.030750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:32.040589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58714944-7f77-4ad8-82b4-8be28f0e4ea3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:32.080092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f8d07-e36e-4cb7-8442-fae7890978b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:32.080693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f8d07-e36e-4cb7-8442-fae7890978b9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.080969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f8d07-e36e-4cb7-8442-fae7890978b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.081268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4f8d07-e36e-4cb7-8442-fae7890978b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.081955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f8d07-e36e-4cb7-8442-fae7890978b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:32.082259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f8d07-e36e-4cb7-8442-fae7890978b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:32.083663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4f8d07-e36e-4cb7-8442-fae7890978b9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:32.084388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edca83d-9a61-40d5-9b34-164136eae738 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:32.095058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 406/808] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:32.095845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f8d07-e36e-4cb7-8442-fae7890978b9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:32.118680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09643c3-1ce7-4380-8506-04c44c293678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:32.119382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09643c3-1ce7-4380-8506-04c44c293678 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.119642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09643c3-1ce7-4380-8506-04c44c293678 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.119913 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09643c3-1ce7-4380-8506-04c44c293678 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.120571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09643c3-1ce7-4380-8506-04c44c293678 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:32.120882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09643c3-1ce7-4380-8506-04c44c293678 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:32.122388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09643c3-1ce7-4380-8506-04c44c293678 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:32.130537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 403/809] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:57:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:32.131166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09643c3-1ce7-4380-8506-04c44c293678 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:32.169374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:32.204500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4f8d07-e36e-4cb7-8442-fae7890978b9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:32.230111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09643c3-1ce7-4380-8506-04c44c293678 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:32.237993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31035a8e-0ebb-42c8-8384-b43c007c224f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:32.238611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31035a8e-0ebb-42c8-8384-b43c007c224f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.238883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31035a8e-0ebb-42c8-8384-b43c007c224f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.239146 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31035a8e-0ebb-42c8-8384-b43c007c224f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.239712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31035a8e-0ebb-42c8-8384-b43c007c224f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:32.241055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31035a8e-0ebb-42c8-8384-b43c007c224f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-31035a8e-0ebb-42c8-8384-b43c007c224f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:32.252272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:32.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 22 21:57:32.254245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:32.257237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 407/810] 10.222.0.25 () {56 vars in 1129 bytes} [Thu Sep 22 21:57:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:32.258623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412e8db-8cb3-4f64-b113-fbe53c62bde2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.258909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412e8db-8cb3-4f64-b113-fbe53c62bde2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.259185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412e8db-8cb3-4f64-b113-fbe53c62bde2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.260448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 408/811] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:32.261143 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412e8db-8cb3-4f64-b113-fbe53c62bde2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:32.263658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a0d2-63ab-44a8-b723-3a7d2def68a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:32.264283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a0d2-63ab-44a8-b723-3a7d2def68a2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.264564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a0d2-63ab-44a8-b723-3a7d2def68a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.264840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a0d2-63ab-44a8-b723-3a7d2def68a2 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.265548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a0d2-63ab-44a8-b723-3a7d2def68a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:32.265854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a0d2-63ab-44a8-b723-3a7d2def68a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:32.267259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a0d2-63ab-44a8-b723-3a7d2def68a2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:32.278732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 404/812] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:32.279526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a0d2-63ab-44a8-b723-3a7d2def68a2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:32.285002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:32.365363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412e8db-8cb3-4f64-b113-fbe53c62bde2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:32.374770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:32.378692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a0d2-63ab-44a8-b723-3a7d2def68a2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:32.399549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bea69-df1b-44de-898a-3335057c7d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:32.400181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bea69-df1b-44de-898a-3335057c7d46 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.400454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bea69-df1b-44de-898a-3335057c7d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.400726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bea69-df1b-44de-898a-3335057c7d46 None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/bf3dde2bfc4240a8af6425778c5eddaf/roles/054c6569cc994d1c950d5e554273033e` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.411742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4863c68-a8f4-47d8-b55d-289552d30a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:32.412352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4863c68-a8f4-47d8-b55d-289552d30a37 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.412627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4863c68-a8f4-47d8-b55d-289552d30a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.412893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4863c68-a8f4-47d8-b55d-289552d30a37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.414468 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bea69-df1b-44de-898a-3335057c7d46 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=bf3dde2bfc4240a8af6425778c5eddaf, role_id=054c6569cc994d1c950d5e554273033e)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:32.416599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601bea69-df1b-44de-898a-3335057c7d46 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:32.424328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4863c68-a8f4-47d8-b55d-289552d30a37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:32.425785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4863c68-a8f4-47d8-b55d-289552d30a37 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:32.428930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 409/813] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:57:32 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/bf3dde2bfc4240a8af6425778c5eddaf/roles/054c6569cc994d1c950d5e554273033e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:57:32.429748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bea69-df1b-44de-898a-3335057c7d46 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e4863c68-a8f4-47d8-b55d-289552d30a37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:32.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:32.442144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:32.443057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:32.443057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:32.443057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:32.443057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 405/814] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:32.443749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee84fd-969b-48dc-bc59-d87105f4ff64 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.443990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee84fd-969b-48dc-bc59-d87105f4ff64 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.444257 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee84fd-969b-48dc-bc59-d87105f4ff64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.464830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3dee84fd-969b-48dc-bc59-d87105f4ff64 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:32.465993 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3dee84fd-969b-48dc-bc59-d87105f4ff64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:32.513890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:32.545166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 406/815] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:32.545166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee84fd-969b-48dc-bc59-d87105f4ff64 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:32.573426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601bea69-df1b-44de-898a-3335057c7d46 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:32.617466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa77757-88d6-455e-844e-e8b897119e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:32.618647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77757-88d6-455e-844e-e8b897119e7c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.619000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77757-88d6-455e-844e-e8b897119e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.619270 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa77757-88d6-455e-844e-e8b897119e7c None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.619914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa77757-88d6-455e-844e-e8b897119e7c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:32.620211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa77757-88d6-455e-844e-e8b897119e7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:32.625177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa77757-88d6-455e-844e-e8b897119e7c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:32.629780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 410/816] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 22 21:57:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:32.630940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa77757-88d6-455e-844e-e8b897119e7c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:32.667028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:32.689308 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee84fd-969b-48dc-bc59-d87105f4ff64 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:32.727986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883061d9-6cd1-4794-a77d-89b6dbee0594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:32.728919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883061d9-6cd1-4794-a77d-89b6dbee0594 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.729494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883061d9-6cd1-4794-a77d-89b6dbee0594 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.729920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883061d9-6cd1-4794-a77d-89b6dbee0594 None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/471e7550ae7a4f909851215d3ed148c6/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.732055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883061d9-6cd1-4794-a77d-89b6dbee0594 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=471e7550ae7a4f909851215d3ed148c6, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:32.733746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883061d9-6cd1-4794-a77d-89b6dbee0594 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:32.753119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 407/817] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:57:32 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/471e7550ae7a4f909851215d3ed148c6/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:57:32.754332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883061d9-6cd1-4794-a77d-89b6dbee0594 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:32.759766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa77757-88d6-455e-844e-e8b897119e7c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:32.830485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:32.842805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f589e0-5b37-4761-bc83-f5f7afc9fcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:32.843436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f589e0-5b37-4761-bc83-f5f7afc9fcfc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.843713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f589e0-5b37-4761-bc83-f5f7afc9fcfc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.843991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f589e0-5b37-4761-bc83-f5f7afc9fcfc None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.844708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f589e0-5b37-4761-bc83-f5f7afc9fcfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:32.845018 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f589e0-5b37-4761-bc83-f5f7afc9fcfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:32.846534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f589e0-5b37-4761-bc83-f5f7afc9fcfc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:32.861865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 411/818] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:32.862866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f589e0-5b37-4761-bc83-f5f7afc9fcfc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:32.900703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883061d9-6cd1-4794-a77d-89b6dbee0594 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:32.950365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c76ad-d80a-4a5d-a2d2-f09a85dc5e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:32.951343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c76ad-d80a-4a5d-a2d2-f09a85dc5e8e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:32.951817 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c76ad-d80a-4a5d-a2d2-f09a85dc5e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:32.952298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c76ad-d80a-4a5d-a2d2-f09a85dc5e8e None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:32.953077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c76ad-d80a-4a5d-a2d2-f09a85dc5e8e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:32.953541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c76ad-d80a-4a5d-a2d2-f09a85dc5e8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:32.955319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018c76ad-d80a-4a5d-a2d2-f09a85dc5e8e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:32.961562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 408/819] 10.222.0.25 () {56 vars in 1124 bytes} [Thu Sep 22 21:57:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:32.962605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c76ad-d80a-4a5d-a2d2-f09a85dc5e8e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:32.970414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:33.002029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f589e0-5b37-4761-bc83-f5f7afc9fcfc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:33.040961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c66ca-53ed-44aa-8374-0df55a5df35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:33.042429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c66ca-53ed-44aa-8374-0df55a5df35c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:33.042899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c66ca-53ed-44aa-8374-0df55a5df35c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:33.043309 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c66ca-53ed-44aa-8374-0df55a5df35c None admin] PATH_INFO: `/v3/services/network` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:33.044090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c66ca-53ed-44aa-8374-0df55a5df35c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:33.045687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c66ca-53ed-44aa-8374-0df55a5df35c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f24c66ca-53ed-44aa-8374-0df55a5df35c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:33.061567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:33.062786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 22 21:57:33.063610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:33.066283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 412/820] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:32 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:33.067354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c66ca-53ed-44aa-8374-0df55a5df35c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:33.096728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c76ad-d80a-4a5d-a2d2-f09a85dc5e8e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:33.127762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:33.143335 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33074f-7c46-43e3-8b5b-bccae318b2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:33.143957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33074f-7c46-43e3-8b5b-bccae318b2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:33.144233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33074f-7c46-43e3-8b5b-bccae318b2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:33.144499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33074f-7c46-43e3-8b5b-bccae318b2f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:33.152473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33074f-7c46-43e3-8b5b-bccae318b2f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:33.153933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33074f-7c46-43e3-8b5b-bccae318b2f7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-da33074f-7c46-43e3-8b5b-bccae318b2f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:33.165961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:33.167024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:33.167960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:33.167960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:33.167960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:33.167960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:33.167960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:33.167960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:33.167960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:33.167960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:33.167960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:57:33.167960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:33.167960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 409/821] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 22 21:57:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:33.168542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33074f-7c46-43e3-8b5b-bccae318b2f7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:33.205661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c66ca-53ed-44aa-8374-0df55a5df35c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:33.235846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:33.252975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3a7c4-73a3-4ddb-a644-97d3d5283efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:33.253623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3a7c4-73a3-4ddb-a644-97d3d5283efd None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:33.253912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3a7c4-73a3-4ddb-a644-97d3d5283efd None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:33.254164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a3a7c4-73a3-4ddb-a644-97d3d5283efd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:33.255218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3a7c4-73a3-4ddb-a644-97d3d5283efd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:33.256914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a3a7c4-73a3-4ddb-a644-97d3d5283efd None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:33.281815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33074f-7c46-43e3-8b5b-bccae318b2f7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:33.287962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 413/822] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 21:57:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:57:33.290042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3a7c4-73a3-4ddb-a644-97d3d5283efd None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:33.387293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedeb6cd-a2ac-4957-9ef4-973023e179a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:33.388471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedeb6cd-a2ac-4957-9ef4-973023e179a2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:33.390502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedeb6cd-a2ac-4957-9ef4-973023e179a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:33.391015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedeb6cd-a2ac-4957-9ef4-973023e179a2 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:33.392151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedeb6cd-a2ac-4957-9ef4-973023e179a2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:33.394096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedeb6cd-a2ac-4957-9ef4-973023e179a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:33.395822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedeb6cd-a2ac-4957-9ef4-973023e179a2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:33.407387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 410/823] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:57:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:33.413097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedeb6cd-a2ac-4957-9ef4-973023e179a2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:33.471442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a3a7c4-73a3-4ddb-a644-97d3d5283efd None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:33.482748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:33.517208 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff89b20-e5e4-4bf9-81d0-cb8648004529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:33.518207 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff89b20-e5e4-4bf9-81d0-cb8648004529 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:33.518686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff89b20-e5e4-4bf9-81d0-cb8648004529 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:33.519521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff89b20-e5e4-4bf9-81d0-cb8648004529 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:33.521640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff89b20-e5e4-4bf9-81d0-cb8648004529 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:33.522171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff89b20-e5e4-4bf9-81d0-cb8648004529 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:33.524622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff89b20-e5e4-4bf9-81d0-cb8648004529 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:33.551007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 414/824] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 22 21:57:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:33.557036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff89b20-e5e4-4bf9-81d0-cb8648004529 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:33.596764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedeb6cd-a2ac-4957-9ef4-973023e179a2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:33.624669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:33.642885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2f450-39ed-482a-8696-ec9934b29df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:33.644223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f450-39ed-482a-8696-ec9934b29df0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:33.644962 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f450-39ed-482a-8696-ec9934b29df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:33.645469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2f450-39ed-482a-8696-ec9934b29df0 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:33.646571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2f450-39ed-482a-8696-ec9934b29df0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:33.647547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2f450-39ed-482a-8696-ec9934b29df0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:33.650329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2f450-39ed-482a-8696-ec9934b29df0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:33.662891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 411/825] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:57:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:33.664849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2f450-39ed-482a-8696-ec9934b29df0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:33.673592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff89b20-e5e4-4bf9-81d0-cb8648004529 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:33.711528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439383a7-e75d-4f29-9028-d86841bebf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:33.712137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439383a7-e75d-4f29-9028-d86841bebf20 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:33.712407 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439383a7-e75d-4f29-9028-d86841bebf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:33.712671 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439383a7-e75d-4f29-9028-d86841bebf20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:33.752646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:33.811678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439383a7-e75d-4f29-9028-d86841bebf20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:33.813792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439383a7-e75d-4f29-9028-d86841bebf20 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:33.826306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2f450-39ed-482a-8696-ec9934b29df0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:33.890152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22775a-bdbf-45a3-9903-0c31cf94875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:33.890764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22775a-bdbf-45a3-9903-0c31cf94875a None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:33.891034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22775a-bdbf-45a3-9903-0c31cf94875a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:33.891299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a22775a-bdbf-45a3-9903-0c31cf94875a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:33.891970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22775a-bdbf-45a3-9903-0c31cf94875a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:33.893359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a22775a-bdbf-45a3-9903-0c31cf94875a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:33.932019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 412/826] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 21:57:33 2022] POST /identity/v3/endpoints => generated 337 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:57:33.969039 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 415/827] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:57:33.975213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22775a-bdbf-45a3-9903-0c31cf94875a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:34.034526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:34.132161 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a22775a-bdbf-45a3-9903-0c31cf94875a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:34.179030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc89288-b162-4fb1-8b54-cba5aa8da79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:34.180075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc89288-b162-4fb1-8b54-cba5aa8da79b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:34.180593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc89288-b162-4fb1-8b54-cba5aa8da79b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:34.181159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc89288-b162-4fb1-8b54-cba5aa8da79b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:34.197172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc89288-b162-4fb1-8b54-cba5aa8da79b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:34.199088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc89288-b162-4fb1-8b54-cba5aa8da79b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-1bc89288-b162-4fb1-8b54-cba5aa8da79b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:34.218007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:34.219003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:34.220099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:34.220099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:34.220099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:34.220099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 413/828] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:34.226343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439383a7-e75d-4f29-9028-d86841bebf20 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:34.307650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:34.344891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439383a7-e75d-4f29-9028-d86841bebf20 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:34.381834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808db8d-ac38-4e72-a6a0-ad0c16718e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:34.382799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808db8d-ac38-4e72-a6a0-ad0c16718e45 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:34.383299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808db8d-ac38-4e72-a6a0-ad0c16718e45 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:34.384589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e808db8d-ac38-4e72-a6a0-ad0c16718e45 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:34.385495 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808db8d-ac38-4e72-a6a0-ad0c16718e45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:34.385976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808db8d-ac38-4e72-a6a0-ad0c16718e45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:34.387526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e808db8d-ac38-4e72-a6a0-ad0c16718e45 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:34.404544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 416/829] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:34.406237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc89288-b162-4fb1-8b54-cba5aa8da79b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:34.526292 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:34.536221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc89288-b162-4fb1-8b54-cba5aa8da79b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:34.575893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefa9da-8a57-4fe9-9b79-44fbdce0bd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:34.576525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa9da-8a57-4fe9-9b79-44fbdce0bd1a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:34.576803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa9da-8a57-4fe9-9b79-44fbdce0bd1a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:34.577072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa9da-8a57-4fe9-9b79-44fbdce0bd1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:34.587256 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefa9da-8a57-4fe9-9b79-44fbdce0bd1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:34.588753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefa9da-8a57-4fe9-9b79-44fbdce0bd1a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-bfefa9da-8a57-4fe9-9b79-44fbdce0bd1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:34.605296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:34.606343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:34.607431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:34.607431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:34.607431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:34.607431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:34.607431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:34.607431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:34.607431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:34.607431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:34.607431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 414/830] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:34.614053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808db8d-ac38-4e72-a6a0-ad0c16718e45 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:34.720998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e808db8d-ac38-4e72-a6a0-ad0c16718e45 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:34.747222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:34.757007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5f559-0c7b-4a97-9de0-0fc9e59a4cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:34.757641 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5f559-0c7b-4a97-9de0-0fc9e59a4cbe None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:34.757918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5f559-0c7b-4a97-9de0-0fc9e59a4cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:34.758702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5f559-0c7b-4a97-9de0-0fc9e59a4cbe None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:34.759401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5f559-0c7b-4a97-9de0-0fc9e59a4cbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:34.759706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5f559-0c7b-4a97-9de0-0fc9e59a4cbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:34.761210 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5f559-0c7b-4a97-9de0-0fc9e59a4cbe None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:34.773633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 417/831] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:34.780558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefa9da-8a57-4fe9-9b79-44fbdce0bd1a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:34.893531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefa9da-8a57-4fe9-9b79-44fbdce0bd1a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:34.931619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060c086-2450-4e13-b566-1a5eaffb64bb None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:34.931815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88177320-f5ea-41a6-9d0b-d3be418be00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:34.931912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060c086-2450-4e13-b566-1a5eaffb64bb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:34.932159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060c086-2450-4e13-b566-1a5eaffb64bb None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:34.932943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177320-f5ea-41a6-9d0b-d3be418be00e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:34.933439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177320-f5ea-41a6-9d0b-d3be418be00e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:34.933760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 418/832] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:57:34.934150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88177320-f5ea-41a6-9d0b-d3be418be00e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:34.935202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177320-f5ea-41a6-9d0b-d3be418be00e None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=d011e54e12674eaeb82c68da628ecb9d) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:34.935709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177320-f5ea-41a6-9d0b-d3be418be00e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:34.937346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88177320-f5ea-41a6-9d0b-d3be418be00e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:34.938938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a82d9-a0db-4dc0-861c-8d6e28cc7e5b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:34.939221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a82d9-a0db-4dc0-861c-8d6e28cc7e5b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:34.939494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a82d9-a0db-4dc0-861c-8d6e28cc7e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:34.949314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 415/833] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:34 2022] GET /identity/v3/role_assignments?user.id=d011e54e12674eaeb82c68da628ecb9d&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:34.963409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c57a82d9-a0db-4dc0-861c-8d6e28cc7e5b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:34.964899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c57a82d9-a0db-4dc0-861c-8d6e28cc7e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:35.043366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 419/834] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:34 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:35.264741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ae0b-71ef-41d3-ad0d-a32b5fba29db None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:35.265280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ae0b-71ef-41d3-ad0d-a32b5fba29db None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:35.265359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e074ae0b-71ef-41d3-ad0d-a32b5fba29db None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:35.266914 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 416/835] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:35.272251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f62bf0-571b-44ee-9ede-8321562ce952 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:35.272657 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f62bf0-571b-44ee-9ede-8321562ce952 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:35.272823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f62bf0-571b-44ee-9ede-8321562ce952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:35.291765 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-53f62bf0-571b-44ee-9ede-8321562ce952 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:35.292962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-53f62bf0-571b-44ee-9ede-8321562ce952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:35.334408 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074ae0b-71ef-41d3-ad0d-a32b5fba29db None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:35.355520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:35.376563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 420/836] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:35 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:35.446713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e074ae0b-71ef-41d3-ad0d-a32b5fba29db None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:35.492067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee4a1c-a48f-4c49-ba22-cce98161c988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:35.493899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee4a1c-a48f-4c49-ba22-cce98161c988 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:35.494475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee4a1c-a48f-4c49-ba22-cce98161c988 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:35.494930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee4a1c-a48f-4c49-ba22-cce98161c988 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:35.506037 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee4a1c-a48f-4c49-ba22-cce98161c988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-00ee4a1c-a48f-4c49-ba22-cce98161c988 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:35.511608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:35.512558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 21:57:35.513416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:35.515613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 417/837] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 21:57:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:35.522279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f62bf0-571b-44ee-9ede-8321562ce952 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:35.571307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:35.633254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f62bf0-571b-44ee-9ede-8321562ce952 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:35.667682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee4a1c-a48f-4c49-ba22-cce98161c988 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:35.670889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec2299-5acc-44cd-b65a-6bd94845c6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:35.671722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec2299-5acc-44cd-b65a-6bd94845c6b4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:35.672003 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec2299-5acc-44cd-b65a-6bd94845c6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:35.672275 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ec2299-5acc-44cd-b65a-6bd94845c6b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:35.672950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec2299-5acc-44cd-b65a-6bd94845c6b4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:35.674179 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec2299-5acc-44cd-b65a-6bd94845c6b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:35.675558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ec2299-5acc-44cd-b65a-6bd94845c6b4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:35.686266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 421/838] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 21:57:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:35.691712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec2299-5acc-44cd-b65a-6bd94845c6b4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:35.757404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:35.783027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee4a1c-a48f-4c49-ba22-cce98161c988 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:35.809185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ec2299-5acc-44cd-b65a-6bd94845c6b4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:35.818812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:35.819493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aae280-97f7-4d22-8627-43d430cace17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:35.820358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aae280-97f7-4d22-8627-43d430cace17 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:35.821580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aae280-97f7-4d22-8627-43d430cace17 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:35.822185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aae280-97f7-4d22-8627-43d430cace17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:35.833733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aae280-97f7-4d22-8627-43d430cace17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-94aae280-97f7-4d22-8627-43d430cace17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:35.839110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:35.840374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:57:35.841638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:35.849369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 418/839] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:35.850474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aae280-97f7-4d22-8627-43d430cace17 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:35.853601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c3734-65db-426c-8334-9c8bbe467794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:35.854586 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c3734-65db-426c-8334-9c8bbe467794 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:35.855072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c3734-65db-426c-8334-9c8bbe467794 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:35.855402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c3734-65db-426c-8334-9c8bbe467794 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:35.868168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c3734-65db-426c-8334-9c8bbe467794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:35.869970 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c3734-65db-426c-8334-9c8bbe467794 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-852c3734-65db-426c-8334-9c8bbe467794 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:35.890480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:35.891475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:35.892370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:35.892370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:35.892370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:35.896245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 422/840] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:35.897118 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c3734-65db-426c-8334-9c8bbe467794 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:35.991003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aae280-97f7-4d22-8627-43d430cace17 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:36.013043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c3734-65db-426c-8334-9c8bbe467794 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:36.026803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba09238-c0fe-43ab-a55a-e06d4effcdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:36.027764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba09238-c0fe-43ab-a55a-e06d4effcdff None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.028048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba09238-c0fe-43ab-a55a-e06d4effcdff None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.028315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba09238-c0fe-43ab-a55a-e06d4effcdff None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.028967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba09238-c0fe-43ab-a55a-e06d4effcdff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:36.029306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba09238-c0fe-43ab-a55a-e06d4effcdff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:36.030664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba09238-c0fe-43ab-a55a-e06d4effcdff None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:36.033145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:36.046015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 419/841] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:57:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:36.046015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753137-b280-4255-9dab-532ae4de75ff None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.046015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753137-b280-4255-9dab-532ae4de75ff None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.046015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68753137-b280-4255-9dab-532ae4de75ff None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.046621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 420/842] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:36.047324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68753137-b280-4255-9dab-532ae4de75ff None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:36.053067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a9797-72a0-451b-a92b-7b39f7bd71ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:36.053777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a9797-72a0-451b-a92b-7b39f7bd71ba None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.054125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a9797-72a0-451b-a92b-7b39f7bd71ba None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.054464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a9797-72a0-451b-a92b-7b39f7bd71ba None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.055810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a9797-72a0-451b-a92b-7b39f7bd71ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:36.056155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a9797-72a0-451b-a92b-7b39f7bd71ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:36.057630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a9797-72a0-451b-a92b-7b39f7bd71ba None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:36.073289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 423/843] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:36.074569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200545a-2f6f-44b2-9c42-1b58e5457fb0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.074861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200545a-2f6f-44b2-9c42-1b58e5457fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.075142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7200545a-2f6f-44b2-9c42-1b58e5457fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.095632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7200545a-2f6f-44b2-9c42-1b58e5457fb0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:36.097029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7200545a-2f6f-44b2-9c42-1b58e5457fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:36.114930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:36.168721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68753137-b280-4255-9dab-532ae4de75ff None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:36.181128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 424/844] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:36.181128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200545a-2f6f-44b2-9c42-1b58e5457fb0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:36.216859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc629f-7af3-48c3-a863-3076825bd316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:36.217502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc629f-7af3-48c3-a863-3076825bd316 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.217778 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc629f-7af3-48c3-a863-3076825bd316 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.218048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc629f-7af3-48c3-a863-3076825bd316 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.231908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc629f-7af3-48c3-a863-3076825bd316 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:36.234478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc629f-7af3-48c3-a863-3076825bd316 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-9dfc629f-7af3-48c3-a863-3076825bd316 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:36.251572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:36.252548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:36.253471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:36.253471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:36.253471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:36.255191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 421/845] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:36.262776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc629f-7af3-48c3-a863-3076825bd316 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:36.297636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7200545a-2f6f-44b2-9c42-1b58e5457fb0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:36.316770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:36.342947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a281b-8a4a-4048-a28d-0d1bfd747dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:36.343582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a281b-8a4a-4048-a28d-0d1bfd747dbe None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.343859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a281b-8a4a-4048-a28d-0d1bfd747dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.344129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a281b-8a4a-4048-a28d-0d1bfd747dbe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.352558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a281b-8a4a-4048-a28d-0d1bfd747dbe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:36.354483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a281b-8a4a-4048-a28d-0d1bfd747dbe None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-650a281b-8a4a-4048-a28d-0d1bfd747dbe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:36.367200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:36.368690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:36.369696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:36.369696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:36.369696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:36.369696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:36.369696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:36.369696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:36.369696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:36.369696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:36.369696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:36.369696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:36.373276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 425/846] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 22 21:57:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:36.374462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a281b-8a4a-4048-a28d-0d1bfd747dbe None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:36.387383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc629f-7af3-48c3-a863-3076825bd316 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:36.388272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:36.439964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d68d0-48a4-4094-b211-ca732e9d56cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:36.441370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d68d0-48a4-4094-b211-ca732e9d56cb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.441610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d68d0-48a4-4094-b211-ca732e9d56cb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.441884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d68d0-48a4-4094-b211-ca732e9d56cb None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.442529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d68d0-48a4-4094-b211-ca732e9d56cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:36.442832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d68d0-48a4-4094-b211-ca732e9d56cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:36.444192 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d68d0-48a4-4094-b211-ca732e9d56cb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:36.460482 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 422/847] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:36.460482 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d68d0-48a4-4094-b211-ca732e9d56cb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:36.495319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a281b-8a4a-4048-a28d-0d1bfd747dbe None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:36.529992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63fbfd0-4f83-4140-98cb-54c4e80e3d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:36.530600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fbfd0-4f83-4140-98cb-54c4e80e3d27 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.530878 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fbfd0-4f83-4140-98cb-54c4e80e3d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.531149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63fbfd0-4f83-4140-98cb-54c4e80e3d27 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.531869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63fbfd0-4f83-4140-98cb-54c4e80e3d27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:36.532175 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63fbfd0-4f83-4140-98cb-54c4e80e3d27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:36.533648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63fbfd0-4f83-4140-98cb-54c4e80e3d27 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:36.552045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 426/848] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 22 21:57:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:36.552685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63fbfd0-4f83-4140-98cb-54c4e80e3d27 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:36.572035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d68d0-48a4-4094-b211-ca732e9d56cb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:36.588897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:36.616208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf73e7-7a92-4262-8a74-41b1cbfcb066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:36.616830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf73e7-7a92-4262-8a74-41b1cbfcb066 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.617118 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf73e7-7a92-4262-8a74-41b1cbfcb066 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.617692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf73e7-7a92-4262-8a74-41b1cbfcb066 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.617992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf73e7-7a92-4262-8a74-41b1cbfcb066 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:36.619552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf73e7-7a92-4262-8a74-41b1cbfcb066 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:36.623027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-9cdf73e7-7a92-4262-8a74-41b1cbfcb066 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:36.628878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:36.629906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:36.630742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 423/849] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:57:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:36.631540 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf73e7-7a92-4262-8a74-41b1cbfcb066 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:36.664268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63fbfd0-4f83-4140-98cb-54c4e80e3d27 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:36.703892 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a014f-f02f-4bb7-a5e2-725913a36a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:36.704454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a014f-f02f-4bb7-a5e2-725913a36a6c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.705061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a014f-f02f-4bb7-a5e2-725913a36a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.705369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a014f-f02f-4bb7-a5e2-725913a36a6c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.714784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a014f-f02f-4bb7-a5e2-725913a36a6c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:36.716568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a014f-f02f-4bb7-a5e2-725913a36a6c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-fa8a014f-f02f-4bb7-a5e2-725913a36a6c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:36.728172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:36.729335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:36.730572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:36.730572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:36.730572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:36.730572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:36.730572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:36.730572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:36.730572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:36.730572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:36.730572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 22 21:57:36.730572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:36.733967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 427/850] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:57:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:36.734783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a014f-f02f-4bb7-a5e2-725913a36a6c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:36.740679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf73e7-7a92-4262-8a74-41b1cbfcb066 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:36.776701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f561138-8e45-435a-a03d-c9bc50bcc084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:36.777477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f561138-8e45-435a-a03d-c9bc50bcc084 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.777824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f561138-8e45-435a-a03d-c9bc50bcc084 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.778101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f561138-8e45-435a-a03d-c9bc50bcc084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.819908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:36.843592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a014f-f02f-4bb7-a5e2-725913a36a6c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:36.848205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f561138-8e45-435a-a03d-c9bc50bcc084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:36.849704 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f561138-8e45-435a-a03d-c9bc50bcc084 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:36.872867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:36.882796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf25a9-7e0d-4bc7-afb9-357c80b9ef20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:36.883468 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf25a9-7e0d-4bc7-afb9-357c80b9ef20 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:36.883733 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf25a9-7e0d-4bc7-afb9-357c80b9ef20 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:36.884005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf25a9-7e0d-4bc7-afb9-357c80b9ef20 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:36.884631 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf25a9-7e0d-4bc7-afb9-357c80b9ef20 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:36.884934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf25a9-7e0d-4bc7-afb9-357c80b9ef20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:36.886344 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf25a9-7e0d-4bc7-afb9-357c80b9ef20 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:36.895257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 428/851] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:57:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:36.895257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf25a9-7e0d-4bc7-afb9-357c80b9ef20 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:36.961269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 424/852] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:36 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:57:36.961269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f561138-8e45-435a-a03d-c9bc50bcc084 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:37.017447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf25a9-7e0d-4bc7-afb9-357c80b9ef20 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:37.038799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:37.066084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e63d03-7ade-49ab-a7e7-fa147da9c1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:37.066592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e63d03-7ade-49ab-a7e7-fa147da9c1a3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:37.066909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e63d03-7ade-49ab-a7e7-fa147da9c1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:37.067247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e63d03-7ade-49ab-a7e7-fa147da9c1a3 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:37.067983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e63d03-7ade-49ab-a7e7-fa147da9c1a3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:37.068288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e63d03-7ade-49ab-a7e7-fa147da9c1a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:37.068961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f561138-8e45-435a-a03d-c9bc50bcc084 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:37.069753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e63d03-7ade-49ab-a7e7-fa147da9c1a3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:37.091533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 429/853] 10.222.0.25 () {56 vars in 1135 bytes} [Thu Sep 22 21:57:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:37.094321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e63d03-7ade-49ab-a7e7-fa147da9c1a3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:37.106774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd205c-9c9c-4617-8728-b342e6f88d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:37.107387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd205c-9c9c-4617-8728-b342e6f88d2e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:37.107658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd205c-9c9c-4617-8728-b342e6f88d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:37.108098 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcd205c-9c9c-4617-8728-b342e6f88d2e None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:37.108561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd205c-9c9c-4617-8728-b342e6f88d2e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:37.108866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd205c-9c9c-4617-8728-b342e6f88d2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:37.110313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcd205c-9c9c-4617-8728-b342e6f88d2e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:37.125152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 425/854] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:57:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:37.125152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd205c-9c9c-4617-8728-b342e6f88d2e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:37.166812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:37.225020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e63d03-7ade-49ab-a7e7-fa147da9c1a3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:37.265828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0e4a3-20cf-4650-9782-083ef8c4df04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:37.266731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e4a3-20cf-4650-9782-083ef8c4df04 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:37.267004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e4a3-20cf-4650-9782-083ef8c4df04 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:37.267307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0e4a3-20cf-4650-9782-083ef8c4df04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:37.268041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcd205c-9c9c-4617-8728-b342e6f88d2e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:37.279874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0e4a3-20cf-4650-9782-083ef8c4df04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:37.281651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde0e4a3-20cf-4650-9782-083ef8c4df04 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bde0e4a3-20cf-4650-9782-083ef8c4df04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:37.297783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:37.299421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:37.300517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:37.300517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:37.300517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:37.300517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 430/855] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:37.300517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7ba11-b6a6-4863-b5bb-c4c1215dda60 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:37.300517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7ba11-b6a6-4863-b5bb-c4c1215dda60 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:37.300918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d7ba11-b6a6-4863-b5bb-c4c1215dda60 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:37.307010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 431/856] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:37.307872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7ba11-b6a6-4863-b5bb-c4c1215dda60 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:37.312578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb4d05-4a0d-4ae7-ac22-237f8cb126ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:37.313227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb4d05-4a0d-4ae7-ac22-237f8cb126ca None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:37.313505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb4d05-4a0d-4ae7-ac22-237f8cb126ca None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:37.313774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb4d05-4a0d-4ae7-ac22-237f8cb126ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:37.323254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:37.384939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb4d05-4a0d-4ae7-ac22-237f8cb126ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:37.386407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb4d05-4a0d-4ae7-ac22-237f8cb126ca None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:37.442098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d7ba11-b6a6-4863-b5bb-c4c1215dda60 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:37.444280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:37.487496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fd764-5d44-45d3-8451-f524cc72772e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:37.488021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fd764-5d44-45d3-8451-f524cc72772e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:37.488296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fd764-5d44-45d3-8451-f524cc72772e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:37.488568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0fd764-5d44-45d3-8451-f524cc72772e None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:37.489264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fd764-5d44-45d3-8451-f524cc72772e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:37.489593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fd764-5d44-45d3-8451-f524cc72772e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:37.490916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0fd764-5d44-45d3-8451-f524cc72772e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:37.496390 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 426/857] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:37 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:57:37.497966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2b8da-03d6-497e-91a8-9aae592a5187 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:37.498426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2b8da-03d6-497e-91a8-9aae592a5187 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:37.498906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2b8da-03d6-497e-91a8-9aae592a5187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:37.506665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 432/858] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:37.507777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fd764-5d44-45d3-8451-f524cc72772e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:37.522764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9ad2b8da-03d6-497e-91a8-9aae592a5187 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:37.524835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9ad2b8da-03d6-497e-91a8-9aae592a5187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:37.556711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:37.597285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 427/859] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:37.597285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2b8da-03d6-497e-91a8-9aae592a5187 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:37.629635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0fd764-5d44-45d3-8451-f524cc72772e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:37.672921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667e056-cb46-49a0-96cd-5a2a55ddfb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:37.673968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667e056-cb46-49a0-96cd-5a2a55ddfb9a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:37.674457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667e056-cb46-49a0-96cd-5a2a55ddfb9a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:37.674972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667e056-cb46-49a0-96cd-5a2a55ddfb9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:37.687406 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667e056-cb46-49a0-96cd-5a2a55ddfb9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:37.689326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667e056-cb46-49a0-96cd-5a2a55ddfb9a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:37.709042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2b8da-03d6-497e-91a8-9aae592a5187 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-3667e056-cb46-49a0-96cd-5a2a55ddfb9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:37.716373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:37.717360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:37.718269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:37.718269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:37.718269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:37.724947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 433/860] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:37.726863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667e056-cb46-49a0-96cd-5a2a55ddfb9a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:37.746051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:37.748245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e88a4-051f-4558-9ed1-69446cafbace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:37.748969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e88a4-051f-4558-9ed1-69446cafbace None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:37.749411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e88a4-051f-4558-9ed1-69446cafbace None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:37.749784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17e88a4-051f-4558-9ed1-69446cafbace None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:37.757658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e88a4-051f-4558-9ed1-69446cafbace None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:37.760013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17e88a4-051f-4558-9ed1-69446cafbace None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a17e88a4-051f-4558-9ed1-69446cafbace None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:37.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:37.773497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:37.775042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:37.775042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:37.775042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:37.775042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:37.775042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:37.775042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:37.775042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:37.775042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:37.775042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 428/861] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:37.780375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e88a4-051f-4558-9ed1-69446cafbace None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:37.823790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:37.851642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667e056-cb46-49a0-96cd-5a2a55ddfb9a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:37.896858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cea1eb-a55b-43b5-b38d-b430d404ea3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:37.897558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cea1eb-a55b-43b5-b38d-b430d404ea3b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:37.897837 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cea1eb-a55b-43b5-b38d-b430d404ea3b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:37.898110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cea1eb-a55b-43b5-b38d-b430d404ea3b None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:37.899053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cea1eb-a55b-43b5-b38d-b430d404ea3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:37.899587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cea1eb-a55b-43b5-b38d-b430d404ea3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:37.901285 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cea1eb-a55b-43b5-b38d-b430d404ea3b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:37.913587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 434/862] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:37.914514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cea1eb-a55b-43b5-b38d-b430d404ea3b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:37.919589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17e88a4-051f-4558-9ed1-69446cafbace None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:37.957216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab99042-94bc-41ab-9cc3-b76f766d63f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:37.958226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab99042-94bc-41ab-9cc3-b76f766d63f3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:37.958712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab99042-94bc-41ab-9cc3-b76f766d63f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:37.959269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab99042-94bc-41ab-9cc3-b76f766d63f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:37.960164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab99042-94bc-41ab-9cc3-b76f766d63f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:37.960669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab99042-94bc-41ab-9cc3-b76f766d63f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:37.962808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab99042-94bc-41ab-9cc3-b76f766d63f3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:37.975703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 429/863] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:37.976623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab99042-94bc-41ab-9cc3-b76f766d63f3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:37.997752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:38.025239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cea1eb-a55b-43b5-b38d-b430d404ea3b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:38.071732 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaeb5c-c0ea-469f-bdd8-0ef597b89f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:38.072372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaeb5c-c0ea-469f-bdd8-0ef597b89f69 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:38.072649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaeb5c-c0ea-469f-bdd8-0ef597b89f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:38.072920 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eaeb5c-c0ea-469f-bdd8-0ef597b89f69 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:38.073547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eaeb5c-c0ea-469f-bdd8-0ef597b89f69 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:38.074910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eaeb5c-c0ea-469f-bdd8-0ef597b89f69 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:38.080467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-48eaeb5c-c0ea-469f-bdd8-0ef597b89f69 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:38.084590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:38.085845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:38.087547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 435/864] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 22 21:57:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:38.088758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaeb5c-c0ea-469f-bdd8-0ef597b89f69 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:38.101088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab99042-94bc-41ab-9cc3-b76f766d63f3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:38.140174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc7ae8-130b-4199-9b9e-4b7dc0517596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:38.140807 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc7ae8-130b-4199-9b9e-4b7dc0517596 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:38.141142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc7ae8-130b-4199-9b9e-4b7dc0517596 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:38.141421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc7ae8-130b-4199-9b9e-4b7dc0517596 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:38.148247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc7ae8-130b-4199-9b9e-4b7dc0517596 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:38.150411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc7ae8-130b-4199-9b9e-4b7dc0517596 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a4dc7ae8-130b-4199-9b9e-4b7dc0517596 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:38.162880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:38.163952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:38.165480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:38.165480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:38.165480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:38.165480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:38.165480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:38.165480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:38.165480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:38.165480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:38.165480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 430/865] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:38.165974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc7ae8-130b-4199-9b9e-4b7dc0517596 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:38.205276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eaeb5c-c0ea-469f-bdd8-0ef597b89f69 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:38.245223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e9df8-dee5-47fc-a657-cbe75014c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:38.246313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e9df8-dee5-47fc-a657-cbe75014c25f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:38.246692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e9df8-dee5-47fc-a657-cbe75014c25f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:38.247066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e9df8-dee5-47fc-a657-cbe75014c25f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:38.248260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e9df8-dee5-47fc-a657-cbe75014c25f None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=bf3dde2bfc4240a8af6425778c5eddaf) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:38.248508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e9df8-dee5-47fc-a657-cbe75014c25f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:38.249945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e9df8-dee5-47fc-a657-cbe75014c25f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:38.261144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 436/866] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:38 2022] GET /identity/v3/role_assignments?user.id=bf3dde2bfc4240a8af6425778c5eddaf&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:38.261144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e9df8-dee5-47fc-a657-cbe75014c25f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:38.264569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:38.283557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc7ae8-130b-4199-9b9e-4b7dc0517596 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:38.338267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56c9da-d956-4565-aa2b-c839586c06ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:38.339027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56c9da-d956-4565-aa2b-c839586c06ac None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:38.339311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56c9da-d956-4565-aa2b-c839586c06ac None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:38.339841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca56c9da-d956-4565-aa2b-c839586c06ac None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:38.340768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56c9da-d956-4565-aa2b-c839586c06ac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:38.341271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56c9da-d956-4565-aa2b-c839586c06ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:38.342814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca56c9da-d956-4565-aa2b-c839586c06ac None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:38.351100 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 431/867] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 22 21:57:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:38.357637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56c9da-d956-4565-aa2b-c839586c06ac None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:38.374017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:38.405911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e9df8-dee5-47fc-a657-cbe75014c25f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:38.446217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bf660-ed73-4678-aeed-cd1ed2bbc807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:38.446843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bf660-ed73-4678-aeed-cd1ed2bbc807 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:38.447101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bf660-ed73-4678-aeed-cd1ed2bbc807 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:38.447368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7bf660-ed73-4678-aeed-cd1ed2bbc807 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:38.448057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bf660-ed73-4678-aeed-cd1ed2bbc807 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:38.448354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bf660-ed73-4678-aeed-cd1ed2bbc807 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:38.450490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7bf660-ed73-4678-aeed-cd1ed2bbc807 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:38.466007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 437/868] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:38.473390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bf660-ed73-4678-aeed-cd1ed2bbc807 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:38.483660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca56c9da-d956-4565-aa2b-c839586c06ac None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:38.528241 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74c42f3-43e1-4179-b0aa-866232dc80d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:38.528853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c42f3-43e1-4179-b0aa-866232dc80d4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:38.529926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c42f3-43e1-4179-b0aa-866232dc80d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:38.529926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c42f3-43e1-4179-b0aa-866232dc80d4 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:38.530147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c42f3-43e1-4179-b0aa-866232dc80d4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:38.530362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c42f3-43e1-4179-b0aa-866232dc80d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:38.531712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74c42f3-43e1-4179-b0aa-866232dc80d4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:38.539863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 432/869] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 22 21:57:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:38.562946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:38.613875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7bf660-ed73-4678-aeed-cd1ed2bbc807 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:38.655051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336497ef-3803-43d5-9e00-99c9bc41d394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:38.655664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336497ef-3803-43d5-9e00-99c9bc41d394 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:38.655941 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336497ef-3803-43d5-9e00-99c9bc41d394 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:38.656212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336497ef-3803-43d5-9e00-99c9bc41d394 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:38.656950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336497ef-3803-43d5-9e00-99c9bc41d394 None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=471e7550ae7a4f909851215d3ed148c6) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:38.657290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336497ef-3803-43d5-9e00-99c9bc41d394 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:38.658729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336497ef-3803-43d5-9e00-99c9bc41d394 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:38.666362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 438/870] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:38 2022] GET /identity/v3/role_assignments?user.id=471e7550ae7a4f909851215d3ed148c6&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:39.415460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08df7f-f116-45c1-82c2-0dfb60a8324b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:39.415959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08df7f-f116-45c1-82c2-0dfb60a8324b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:39.415959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc08df7f-f116-45c1-82c2-0dfb60a8324b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:39.417582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 433/871] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:39.423201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c71db-cb50-4064-b23c-f8109fdd7db8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:39.423640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c71db-cb50-4064-b23c-f8109fdd7db8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:39.423741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201c71db-cb50-4064-b23c-f8109fdd7db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:39.443321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-201c71db-cb50-4064-b23c-f8109fdd7db8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:39.444740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-201c71db-cb50-4064-b23c-f8109fdd7db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:39.530645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 439/872] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:39.805712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08df7f-f116-45c1-82c2-0dfb60a8324b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:39.898162 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:39.930929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc08df7f-f116-45c1-82c2-0dfb60a8324b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:39.970932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f24094-eecc-4cdb-9db5-e62469255b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:39.971892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f24094-eecc-4cdb-9db5-e62469255b60 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:39.972344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f24094-eecc-4cdb-9db5-e62469255b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:39.972808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f24094-eecc-4cdb-9db5-e62469255b60 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:39.973911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f24094-eecc-4cdb-9db5-e62469255b60 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:39.975726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f24094-eecc-4cdb-9db5-e62469255b60 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:39.993244 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 434/873] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:57:39 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:57:40.756918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2807b2-bdfb-4327-a054-a5a1a87485a3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:40.758585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2807b2-bdfb-4327-a054-a5a1a87485a3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:40.759071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2807b2-bdfb-4327-a054-a5a1a87485a3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:40.761233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 440/874] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:40.767358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158b273-e91f-4ca2-997a-e8746632be19 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:40.768246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158b273-e91f-4ca2-997a-e8746632be19 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:40.768737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158b273-e91f-4ca2-997a-e8746632be19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:40.788883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a158b273-e91f-4ca2-997a-e8746632be19 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:40.791131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a158b273-e91f-4ca2-997a-e8746632be19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:40.866173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 435/875] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:57:40.879534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54ff7-d671-4e8f-9e91-1c1cbbf97a18 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:40.880229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54ff7-d671-4e8f-9e91-1c1cbbf97a18 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:40.880702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e54ff7-d671-4e8f-9e91-1c1cbbf97a18 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:40.882510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 441/876] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:40.890663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e29e-faac-46ba-9e98-45898d4bee18 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:40.891221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e29e-faac-46ba-9e98-45898d4bee18 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:40.891705 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52e29e-faac-46ba-9e98-45898d4bee18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:40.910307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8b52e29e-faac-46ba-9e98-45898d4bee18 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:40.911787 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8b52e29e-faac-46ba-9e98-45898d4bee18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:40.977522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:40.985920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 436/877] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:40 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:41.062256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58532f3-4dee-4d5f-acaf-16fa97cc778e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:41.062798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58532f3-4dee-4d5f-acaf-16fa97cc778e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:41.063279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58532f3-4dee-4d5f-acaf-16fa97cc778e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:41.065068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 442/878] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:41.071333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680faaeb-9193-4bf8-970e-b5a1d317ded3 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:41.071621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680faaeb-9193-4bf8-970e-b5a1d317ded3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:41.071897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680faaeb-9193-4bf8-970e-b5a1d317ded3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:41.090470 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-680faaeb-9193-4bf8-970e-b5a1d317ded3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:41.091657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-680faaeb-9193-4bf8-970e-b5a1d317ded3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:41.162056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 437/879] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:41 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:41.180169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58532f3-4dee-4d5f-acaf-16fa97cc778e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:41.238283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:41.279261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680faaeb-9193-4bf8-970e-b5a1d317ded3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:41.290572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58532f3-4dee-4d5f-acaf-16fa97cc778e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:41.326974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bd78b-2334-46eb-8c8d-621f51e103ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:41.328178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bd78b-2334-46eb-8c8d-621f51e103ca None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:41.328657 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bd78b-2334-46eb-8c8d-621f51e103ca None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:41.329168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bd78b-2334-46eb-8c8d-621f51e103ca None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:41.340709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94bd78b-2334-46eb-8c8d-621f51e103ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f94bd78b-2334-46eb-8c8d-621f51e103ca None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:41.346109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:41.347122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:57:41.348021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:41.356067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 443/880] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 22 21:57:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:41.358035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bd78b-2334-46eb-8c8d-621f51e103ca None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:41.390583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680faaeb-9193-4bf8-970e-b5a1d317ded3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:41.419963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:41.435814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40eb44-c207-4e70-8495-f329f5cfb3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:41.436568 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40eb44-c207-4e70-8495-f329f5cfb3ab None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:41.436847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40eb44-c207-4e70-8495-f329f5cfb3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:41.437142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e40eb44-c207-4e70-8495-f329f5cfb3ab None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:41.437802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40eb44-c207-4e70-8495-f329f5cfb3ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:41.440088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e40eb44-c207-4e70-8495-f329f5cfb3ab None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:41.465160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 438/881] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:57:41 2022] POST /identity/v3/services => generated 269 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:57:41.465160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40eb44-c207-4e70-8495-f329f5cfb3ab None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:41.486226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bd78b-2334-46eb-8c8d-621f51e103ca None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:41.512746 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:41.547596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303056e-5ca5-4bd7-bf2e-9f084131c93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:41.549218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303056e-5ca5-4bd7-bf2e-9f084131c93f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:41.549502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303056e-5ca5-4bd7-bf2e-9f084131c93f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:41.549773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303056e-5ca5-4bd7-bf2e-9f084131c93f None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:41.550447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303056e-5ca5-4bd7-bf2e-9f084131c93f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:41.550758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303056e-5ca5-4bd7-bf2e-9f084131c93f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:41.552149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303056e-5ca5-4bd7-bf2e-9f084131c93f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:41.565454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 444/882] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 22 21:57:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:41.571190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303056e-5ca5-4bd7-bf2e-9f084131c93f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:41.594713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e40eb44-c207-4e70-8495-f329f5cfb3ab None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:41.634673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def638f1-e80d-4908-9f43-68f0402b0172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:41.635725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def638f1-e80d-4908-9f43-68f0402b0172 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:41.636261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def638f1-e80d-4908-9f43-68f0402b0172 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:41.636746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def638f1-e80d-4908-9f43-68f0402b0172 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:41.637484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def638f1-e80d-4908-9f43-68f0402b0172 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:41.638945 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def638f1-e80d-4908-9f43-68f0402b0172 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-def638f1-e80d-4908-9f43-68f0402b0172 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:41.649068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:41.650201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:41.651047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 439/883] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 22 21:57:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:41.659980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def638f1-e80d-4908-9f43-68f0402b0172 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:41.696517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303056e-5ca5-4bd7-bf2e-9f084131c93f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:41.717869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:41.750763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f750fe6-9c0a-48c2-986f-705565ba9fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:41.751489 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750fe6-9c0a-48c2-986f-705565ba9fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:41.751829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750fe6-9c0a-48c2-986f-705565ba9fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:41.752165 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f750fe6-9c0a-48c2-986f-705565ba9fe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:41.763995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f750fe6-9c0a-48c2-986f-705565ba9fe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:41.766230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f750fe6-9c0a-48c2-986f-705565ba9fe1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1f750fe6-9c0a-48c2-986f-705565ba9fe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:41.783101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:41.784333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:41.785424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:41.785424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:41.785424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:41.785609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def638f1-e80d-4908-9f43-68f0402b0172 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:41.786014 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 445/884] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:41.794092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f750fe6-9c0a-48c2-986f-705565ba9fe1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:41.829768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fc879-c522-43ab-8e9f-399dddde5c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:41.830746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fc879-c522-43ab-8e9f-399dddde5c15 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:41.831233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fc879-c522-43ab-8e9f-399dddde5c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:41.831710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759fc879-c522-43ab-8e9f-399dddde5c15 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:41.832060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:41.833150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fc879-c522-43ab-8e9f-399dddde5c15 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:41.833675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fc879-c522-43ab-8e9f-399dddde5c15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:41.835242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759fc879-c522-43ab-8e9f-399dddde5c15 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:41.841384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 440/885] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 22 21:57:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:41.849610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fc879-c522-43ab-8e9f-399dddde5c15 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:41.931308 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f750fe6-9c0a-48c2-986f-705565ba9fe1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:41.971452 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc5640-0c1e-4528-b28b-f91478ae3174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:41.972327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc5640-0c1e-4528-b28b-f91478ae3174 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:41.972986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc5640-0c1e-4528-b28b-f91478ae3174 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:41.973833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cc5640-0c1e-4528-b28b-f91478ae3174 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:41.974899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc5640-0c1e-4528-b28b-f91478ae3174 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:41.975357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc5640-0c1e-4528-b28b-f91478ae3174 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:41.976263 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759fc879-c522-43ab-8e9f-399dddde5c15 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:41.977763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cc5640-0c1e-4528-b28b-f91478ae3174 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:41.992976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:42.001516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 446/886] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:42.003418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc5640-0c1e-4528-b28b-f91478ae3174 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:42.028026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513d179-31e1-47e2-b1e9-5f66adf4d44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:42.028636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513d179-31e1-47e2-b1e9-5f66adf4d44b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:42.029340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513d179-31e1-47e2-b1e9-5f66adf4d44b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:42.029624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d513d179-31e1-47e2-b1e9-5f66adf4d44b None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:42.030263 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513d179-31e1-47e2-b1e9-5f66adf4d44b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:42.030567 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513d179-31e1-47e2-b1e9-5f66adf4d44b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:42.031928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d513d179-31e1-47e2-b1e9-5f66adf4d44b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:42.041298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 441/887] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 22 21:57:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:42.130251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:42.149674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cc5640-0c1e-4528-b28b-f91478ae3174 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:42.186871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a5209-2414-4518-bb04-7774c0efd551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:42.187510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a5209-2414-4518-bb04-7774c0efd551 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:42.187780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a5209-2414-4518-bb04-7774c0efd551 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:42.188046 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a5209-2414-4518-bb04-7774c0efd551 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:42.196746 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a5209-2414-4518-bb04-7774c0efd551 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:42.198276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a5209-2414-4518-bb04-7774c0efd551 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-049a5209-2414-4518-bb04-7774c0efd551 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:42.216127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:42.217265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:42.218209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:42.218209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:42.218209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:42.218209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:42.218209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:42.218209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:42.218209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:42.218209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:42.218209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:42.218209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:42.220968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 447/888] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 22 21:57:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:42.230364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513d179-31e1-47e2-b1e9-5f66adf4d44b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:42.345261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d513d179-31e1-47e2-b1e9-5f66adf4d44b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:42.376149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f190a-b780-448f-8956-60edbf737540 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:42.376464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f190a-b780-448f-8956-60edbf737540 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:42.376710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794f190a-b780-448f-8956-60edbf737540 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:42.379428 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 448/889] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:57:42.383137 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc083f1-5f6c-491f-b219-57f569ae65cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:42.384023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac467-27ad-41b6-825c-c32d1e19c4c9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:42.384362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac467-27ad-41b6-825c-c32d1e19c4c9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:42.384477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc083f1-5f6c-491f-b219-57f569ae65cb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:42.384684 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dac467-27ad-41b6-825c-c32d1e19c4c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:42.385186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc083f1-5f6c-491f-b219-57f569ae65cb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:42.385668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc083f1-5f6c-491f-b219-57f569ae65cb None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:42.386598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc083f1-5f6c-491f-b219-57f569ae65cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:42.387107 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc083f1-5f6c-491f-b219-57f569ae65cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:42.388743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc083f1-5f6c-491f-b219-57f569ae65cb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:42.404090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-35dac467-27ad-41b6-825c-c32d1e19c4c9 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:42.405329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-35dac467-27ad-41b6-825c-c32d1e19c4c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:42.411906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 442/890] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 22 21:57:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:42.418853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc083f1-5f6c-491f-b219-57f569ae65cb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:42.460302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:42.491947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 449/891] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:42 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:42.546900 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc083f1-5f6c-491f-b219-57f569ae65cb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:42.582863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a0f46-c03c-4ecb-bfd8-f4f571243153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:42.583855 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a0f46-c03c-4ecb-bfd8-f4f571243153 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:42.584344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a0f46-c03c-4ecb-bfd8-f4f571243153 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:42.584846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620a0f46-c03c-4ecb-bfd8-f4f571243153 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:42.655813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a0f46-c03c-4ecb-bfd8-f4f571243153 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:42.657589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620a0f46-c03c-4ecb-bfd8-f4f571243153 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:42.687924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:42.764111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 443/892] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:42 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 21:57:42.769528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac467-27ad-41b6-825c-c32d1e19c4c9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:42.780508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a0f46-c03c-4ecb-bfd8-f4f571243153 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:42.858904 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:42.884383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dac467-27ad-41b6-825c-c32d1e19c4c9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:42.895254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620a0f46-c03c-4ecb-bfd8-f4f571243153 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:42.924693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5baa19-0f1d-4894-8e46-7d0e7725c8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:42.925301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5baa19-0f1d-4894-8e46-7d0e7725c8fe None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:42.925831 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5baa19-0f1d-4894-8e46-7d0e7725c8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:42.925831 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5baa19-0f1d-4894-8e46-7d0e7725c8fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:42.929291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:42.937287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a3966-92ea-47d2-8fe0-9342e2c798db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:42.938154 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a3966-92ea-47d2-8fe0-9342e2c798db None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:42.938419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5baa19-0f1d-4894-8e46-7d0e7725c8fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:42.938748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a3966-92ea-47d2-8fe0-9342e2c798db None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:42.939177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282a3966-92ea-47d2-8fe0-9342e2c798db None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:42.939871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5baa19-0f1d-4894-8e46-7d0e7725c8fe None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:42.940071 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a3966-92ea-47d2-8fe0-9342e2c798db None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:42.941691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282a3966-92ea-47d2-8fe0-9342e2c798db None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-282a3966-92ea-47d2-8fe0-9342e2c798db None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:42.951850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:42.953128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 21:57:42.954153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5e5baa19-0f1d-4894-8e46-7d0e7725c8fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:42.955085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:42.955961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:42.956926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:42.956926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:42.956926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:42.958416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 444/894] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:57:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:42.961140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 450/894] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:42.962109 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a3966-92ea-47d2-8fe0-9342e2c798db None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:42.965353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5baa19-0f1d-4894-8e46-7d0e7725c8fe None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:43.086466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282a3966-92ea-47d2-8fe0-9342e2c798db None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:43.091529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5baa19-0f1d-4894-8e46-7d0e7725c8fe None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:43.119904 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:43.129276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53c850-5bf4-4a30-932d-f95b682e957e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:43.129938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53c850-5bf4-4a30-932d-f95b682e957e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:43.130160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53c850-5bf4-4a30-932d-f95b682e957e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:43.130436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be53c850-5bf4-4a30-932d-f95b682e957e None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:43.131064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53c850-5bf4-4a30-932d-f95b682e957e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:43.131374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53c850-5bf4-4a30-932d-f95b682e957e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:43.132916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be53c850-5bf4-4a30-932d-f95b682e957e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:43.136131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f1be5-23cc-4e27-b710-116113657e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:43.137292 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f1be5-23cc-4e27-b710-116113657e5e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:43.137789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f1be5-23cc-4e27-b710-116113657e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:43.138310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f1be5-23cc-4e27-b710-116113657e5e None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:43.138553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 451/895] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:57:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:43.139425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f1be5-23cc-4e27-b710-116113657e5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:43.139941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f1be5-23cc-4e27-b710-116113657e5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:43.141621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f1be5-23cc-4e27-b710-116113657e5e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:43.149883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53c850-5bf4-4a30-932d-f95b682e957e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:43.160791 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 445/896] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:43.163289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f1be5-23cc-4e27-b710-116113657e5e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:43.230069 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:43.265670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be53c850-5bf4-4a30-932d-f95b682e957e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:43.281868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f1be5-23cc-4e27-b710-116113657e5e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:43.302417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390b1c5-8f13-480d-8564-6529a9ea41ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:43.303321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390b1c5-8f13-480d-8564-6529a9ea41ee None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:43.303782 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390b1c5-8f13-480d-8564-6529a9ea41ee None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:43.304226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e390b1c5-8f13-480d-8564-6529a9ea41ee None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:43.305261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390b1c5-8f13-480d-8564-6529a9ea41ee None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:43.305780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390b1c5-8f13-480d-8564-6529a9ea41ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:43.307356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e390b1c5-8f13-480d-8564-6529a9ea41ee None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:43.317211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 452/897] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:57:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:43.319178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390b1c5-8f13-480d-8564-6529a9ea41ee None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:43.323907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010f9f3-a351-496e-a861-efadcac450b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:43.324553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010f9f3-a351-496e-a861-efadcac450b8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:43.324831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010f9f3-a351-496e-a861-efadcac450b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:43.325118 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8010f9f3-a351-496e-a861-efadcac450b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:43.333068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010f9f3-a351-496e-a861-efadcac450b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:43.335259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8010f9f3-a351-496e-a861-efadcac450b8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-8010f9f3-a351-496e-a861-efadcac450b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:43.351403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:43.352975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:43.353898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:43.353898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:43.353898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:43.353898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:43.353898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:43.353898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:43.353898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:43.353898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:43.359512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 446/898] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:43.359870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010f9f3-a351-496e-a861-efadcac450b8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:43.393385 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:43.440857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e390b1c5-8f13-480d-8564-6529a9ea41ee None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:43.477857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a9777-8b85-41ce-8fce-e20c8d520591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:43.478496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a9777-8b85-41ce-8fce-e20c8d520591 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:43.478772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a9777-8b85-41ce-8fce-e20c8d520591 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:43.479038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398a9777-8b85-41ce-8fce-e20c8d520591 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:43.479731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a9777-8b85-41ce-8fce-e20c8d520591 None admin] RBAC: Adding query filter params (interface=public, service_id=07abeb6f40774bf4b27e14e7d3adc243, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:43.480031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a9777-8b85-41ce-8fce-e20c8d520591 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:43.481465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398a9777-8b85-41ce-8fce-e20c8d520591 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:43.487246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8010f9f3-a351-496e-a861-efadcac450b8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:43.491333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 453/899] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 22 21:57:43 2022] GET /identity/v3/endpoints?service_id=07abeb6f40774bf4b27e14e7d3adc243&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:43.493348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a9777-8b85-41ce-8fce-e20c8d520591 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:43.506595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:43.527793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ac6ca-a32b-48c4-9da4-c644ef553256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:43.528845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ac6ca-a32b-48c4-9da4-c644ef553256 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:43.529361 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ac6ca-a32b-48c4-9da4-c644ef553256 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:43.529832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ac6ca-a32b-48c4-9da4-c644ef553256 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:43.530744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ac6ca-a32b-48c4-9da4-c644ef553256 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:43.531245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ac6ca-a32b-48c4-9da4-c644ef553256 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:43.533050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36ac6ca-a32b-48c4-9da4-c644ef553256 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:43.551878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 447/900] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:43.553215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ac6ca-a32b-48c4-9da4-c644ef553256 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:43.605254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398a9777-8b85-41ce-8fce-e20c8d520591 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:43.627528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:43.644110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d11690-c71f-435e-8d1b-4d3baa97e0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:43.645049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d11690-c71f-435e-8d1b-4d3baa97e0ee None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:43.645181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d11690-c71f-435e-8d1b-4d3baa97e0ee None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:43.645391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d11690-c71f-435e-8d1b-4d3baa97e0ee None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:43.646047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d11690-c71f-435e-8d1b-4d3baa97e0ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:43.647465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d11690-c71f-435e-8d1b-4d3baa97e0ee None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:43.657275 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 454/901] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 22 21:57:43 2022] GET /identity/v3/services => generated 1268 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 21:57:43.665789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ac6ca-a32b-48c4-9da4-c644ef553256 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:43.705980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419f4c0-9073-4efe-9d15-ec4a94b82b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:43.707012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419f4c0-9073-4efe-9d15-ec4a94b82b80 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:43.707491 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419f4c0-9073-4efe-9d15-ec4a94b82b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:43.707961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419f4c0-9073-4efe-9d15-ec4a94b82b80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:43.709029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419f4c0-9073-4efe-9d15-ec4a94b82b80 None admin] RBAC: Adding query filter params (role.id=f1c696334f084cccb674b2528a9fa65c, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=bf3dde2bfc4240a8af6425778c5eddaf) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:43.709507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419f4c0-9073-4efe-9d15-ec4a94b82b80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:43.711309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419f4c0-9073-4efe-9d15-ec4a94b82b80 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:43.718545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 448/902] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:43 2022] GET /identity/v3/role_assignments?user.id=bf3dde2bfc4240a8af6425778c5eddaf&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=f1c696334f084cccb674b2528a9fa65c => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:43.880585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af982cde-d03f-48c8-abca-1046b45e08ac None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:43.881188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af982cde-d03f-48c8-abca-1046b45e08ac None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:43.881188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af982cde-d03f-48c8-abca-1046b45e08ac None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:43.883391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 455/903] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:57:43.889792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ae2a7-222c-49cc-b9b7-5a6af1975460 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:43.890444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ae2a7-222c-49cc-b9b7-5a6af1975460 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:43.890934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808ae2a7-222c-49cc-b9b7-5a6af1975460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:43.910347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-808ae2a7-222c-49cc-b9b7-5a6af1975460 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:43.912319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-808ae2a7-222c-49cc-b9b7-5a6af1975460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:43.951018 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:43.987034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 449/904] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:43 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:44.259836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af982cde-d03f-48c8-abca-1046b45e08ac None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:44.369810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af982cde-d03f-48c8-abca-1046b45e08ac None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:44.405356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013e3cd-debb-464c-87b9-b84bb396d4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:44.406076 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013e3cd-debb-464c-87b9-b84bb396d4ab None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:44.406353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013e3cd-debb-464c-87b9-b84bb396d4ab None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:44.406622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5013e3cd-debb-464c-87b9-b84bb396d4ab None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:44.407200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013e3cd-debb-464c-87b9-b84bb396d4ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:44.407458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a991a-ea93-4455-ac15-3b85d7787792 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:44.407749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a991a-ea93-4455-ac15-3b85d7787792 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:44.408028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a991a-ea93-4455-ac15-3b85d7787792 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:44.408547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5013e3cd-debb-464c-87b9-b84bb396d4ab None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:44.409667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 450/905] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:57:44.415229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067027a-d817-4514-9bc5-015481b9df83 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:44.415512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067027a-d817-4514-9bc5-015481b9df83 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:44.415792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2067027a-d817-4514-9bc5-015481b9df83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5013e3cd-debb-464c-87b9-b84bb396d4ab None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:44.417908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:44.418828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:44.419702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 456/906] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 22 21:57:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:44.424787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013e3cd-debb-464c-87b9-b84bb396d4ab None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:44.438788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2067027a-d817-4514-9bc5-015481b9df83 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:44.440407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2067027a-d817-4514-9bc5-015481b9df83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:44.475592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:44.519057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 451/907] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:57:44.528383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5013e3cd-debb-464c-87b9-b84bb396d4ab None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:44.563733 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ff8f6-d0e5-4464-a1dc-680248a71a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:44.564330 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ff8f6-d0e5-4464-a1dc-680248a71a13 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:44.564603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ff8f6-d0e5-4464-a1dc-680248a71a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:44.564868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ff8f6-d0e5-4464-a1dc-680248a71a13 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:44.565512 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ff8f6-d0e5-4464-a1dc-680248a71a13 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:44.565810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ff8f6-d0e5-4464-a1dc-680248a71a13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:44.567155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ff8f6-d0e5-4464-a1dc-680248a71a13 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:44.576516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 457/908] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 22 21:57:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:44.577910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067027a-d817-4514-9bc5-015481b9df83 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:44.685021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:44.689481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2067027a-d817-4514-9bc5-015481b9df83 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:44.728709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05681c8a-b885-4661-970d-4e89a5be085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:44.729698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05681c8a-b885-4661-970d-4e89a5be085c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:44.730200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05681c8a-b885-4661-970d-4e89a5be085c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:44.730668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05681c8a-b885-4661-970d-4e89a5be085c None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:44.731494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05681c8a-b885-4661-970d-4e89a5be085c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:44.732002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05681c8a-b885-4661-970d-4e89a5be085c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:44.733601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05681c8a-b885-4661-970d-4e89a5be085c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:44.739015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 452/909] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 22 21:57:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:44.744613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ff8f6-d0e5-4464-a1dc-680248a71a13 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:44.792598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05681c8a-b885-4661-970d-4e89a5be085c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:44.854361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ff8f6-d0e5-4464-a1dc-680248a71a13 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:44.869958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:44.894530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4e049-7ccd-4556-9957-e272baff96f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:44.895161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4e049-7ccd-4556-9957-e272baff96f6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:44.895499 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4e049-7ccd-4556-9957-e272baff96f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:44.895931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4e049-7ccd-4556-9957-e272baff96f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:44.896789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4e049-7ccd-4556-9957-e272baff96f6 None admin] RBAC: Adding query filter params (interface=public, service_id=b46b7714913f4bde91a441a8c290fdbe, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:44.897234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4e049-7ccd-4556-9957-e272baff96f6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:44.898837 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4e049-7ccd-4556-9957-e272baff96f6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:44.902988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05681c8a-b885-4661-970d-4e89a5be085c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:44.904639 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 458/910] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 22 21:57:44 2022] GET /identity/v3/endpoints?service_id=b46b7714913f4bde91a441a8c290fdbe&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:44.910022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4e049-7ccd-4556-9957-e272baff96f6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:44.941688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b424fb-01e7-4d95-b2b7-fda1c9b616fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:44.942390 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b424fb-01e7-4d95-b2b7-fda1c9b616fb None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:44.942673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b424fb-01e7-4d95-b2b7-fda1c9b616fb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:44.943044 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b424fb-01e7-4d95-b2b7-fda1c9b616fb None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:44.943711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b424fb-01e7-4d95-b2b7-fda1c9b616fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:44.945828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b424fb-01e7-4d95-b2b7-fda1c9b616fb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:44.965392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 453/911] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:57:44 2022] POST /identity/v3/services => generated 241 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:57:45.034316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4e049-7ccd-4556-9957-e272baff96f6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:45.079002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aace97-563a-40e5-a7d8-7ad180f487ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:45.079677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aace97-563a-40e5-a7d8-7ad180f487ce None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:45.079997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aace97-563a-40e5-a7d8-7ad180f487ce None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:45.080267 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aace97-563a-40e5-a7d8-7ad180f487ce None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:45.080887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aace97-563a-40e5-a7d8-7ad180f487ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:45.082390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aace97-563a-40e5-a7d8-7ad180f487ce None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:45.089736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 459/912] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 22 21:57:44 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 21:57:45.894783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ef6c8-804b-4fb4-b8b4-afaeee147131 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:45.895186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ef6c8-804b-4fb4-b8b4-afaeee147131 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:45.895384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9ef6c8-804b-4fb4-b8b4-afaeee147131 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:45.896967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 454/913] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:45.903164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bee94-2ce8-4fde-9117-51a6f4dd0aea None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:45.903689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bee94-2ce8-4fde-9117-51a6f4dd0aea None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:45.903689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bee94-2ce8-4fde-9117-51a6f4dd0aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:45.924105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ad3bee94-2ce8-4fde-9117-51a6f4dd0aea None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:45.925360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ad3bee94-2ce8-4fde-9117-51a6f4dd0aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:45.958481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:46.004372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 460/914] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:46.038474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b985-ad49-4e4a-80a6-e52b7c85314a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:46.039140 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b985-ad49-4e4a-80a6-e52b7c85314a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:46.039418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d31b985-ad49-4e4a-80a6-e52b7c85314a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:46.040997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 455/915] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:46.046304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86c8d9-095c-47b9-b128-b1f8625574ad None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:46.046583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86c8d9-095c-47b9-b128-b1f8625574ad None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:46.046851 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd86c8d9-095c-47b9-b128-b1f8625574ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:46.076932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-dd86c8d9-095c-47b9-b128-b1f8625574ad None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:46.078387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dd86c8d9-095c-47b9-b128-b1f8625574ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:46.148349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 461/916] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:46.285292 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31b985-ad49-4e4a-80a6-e52b7c85314a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:46.313829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:46.396396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d31b985-ad49-4e4a-80a6-e52b7c85314a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:46.419491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86c8d9-095c-47b9-b128-b1f8625574ad None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:46.432903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e06518-dcbb-468a-a42f-ea078303d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:46.433841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e06518-dcbb-468a-a42f-ea078303d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:46.434320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e06518-dcbb-468a-a42f-ea078303d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:46.434802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e06518-dcbb-468a-a42f-ea078303d7e7 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:46.435387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e06518-dcbb-468a-a42f-ea078303d7e7 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:46.436751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e06518-dcbb-468a-a42f-ea078303d7e7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-77e06518-dcbb-468a-a42f-ea078303d7e7 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:46.446410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:46.447397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:46.448235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 456/917] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:57:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:46.453457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e06518-dcbb-468a-a42f-ea078303d7e7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:46.543212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:46.549534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd86c8d9-095c-47b9-b128-b1f8625574ad None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:46.563289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:46.580426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e06518-dcbb-468a-a42f-ea078303d7e7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:46.587157 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fe9a0-edb1-42e7-a666-0c5806e1e8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:46.587833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fe9a0-edb1-42e7-a666-0c5806e1e8c1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:46.588107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fe9a0-edb1-42e7-a666-0c5806e1e8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:46.588430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493fe9a0-edb1-42e7-a666-0c5806e1e8c1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:46.599738 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493fe9a0-edb1-42e7-a666-0c5806e1e8c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-493fe9a0-edb1-42e7-a666-0c5806e1e8c1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:46.605548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:46.606496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:46.607341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 462/918] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 22 21:57:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:46.613698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fe9a0-edb1-42e7-a666-0c5806e1e8c1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:46.629758 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaeafc2-af37-44b5-9eba-4a8ed44d20e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:46.630617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaeafc2-af37-44b5-9eba-4a8ed44d20e5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:46.631297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaeafc2-af37-44b5-9eba-4a8ed44d20e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:46.631715 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaeafc2-af37-44b5-9eba-4a8ed44d20e5 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:46.633228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaeafc2-af37-44b5-9eba-4a8ed44d20e5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:46.633814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaeafc2-af37-44b5-9eba-4a8ed44d20e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:46.635744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaeafc2-af37-44b5-9eba-4a8ed44d20e5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:46.642725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 457/919] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:57:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:46.648418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaeafc2-af37-44b5-9eba-4a8ed44d20e5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:46.735363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493fe9a0-edb1-42e7-a666-0c5806e1e8c1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:46.766484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaeafc2-af37-44b5-9eba-4a8ed44d20e5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:46.772780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8840b64-0e86-46b0-a35e-637b4c6ca0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:46.773734 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840b64-0e86-46b0-a35e-637b4c6ca0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:46.774222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840b64-0e86-46b0-a35e-637b4c6ca0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:46.774747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8840b64-0e86-46b0-a35e-637b4c6ca0c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:46.775607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8840b64-0e86-46b0-a35e-637b4c6ca0c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:46.776127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8840b64-0e86-46b0-a35e-637b4c6ca0c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:46.777870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8840b64-0e86-46b0-a35e-637b4c6ca0c9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:46.787261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 463/920] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 22 21:57:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:46.794177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8840b64-0e86-46b0-a35e-637b4c6ca0c9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:46.812279 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d188027-8ab3-4bd5-b26a-93f25fb56b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:46.812994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d188027-8ab3-4bd5-b26a-93f25fb56b8b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:46.813531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d188027-8ab3-4bd5-b26a-93f25fb56b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:46.813863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d188027-8ab3-4bd5-b26a-93f25fb56b8b None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:46.814495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d188027-8ab3-4bd5-b26a-93f25fb56b8b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:46.814799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d188027-8ab3-4bd5-b26a-93f25fb56b8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:46.816433 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d188027-8ab3-4bd5-b26a-93f25fb56b8b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:46.821293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:46.828566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 458/921] 10.222.0.25 () {56 vars in 1114 bytes} [Thu Sep 22 21:57:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:46.829504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d188027-8ab3-4bd5-b26a-93f25fb56b8b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:46.853883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:46.926542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8840b64-0e86-46b0-a35e-637b4c6ca0c9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:46.950537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d188027-8ab3-4bd5-b26a-93f25fb56b8b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:46.964277 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cd6e8-a69a-419f-947e-c4c84b00bbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:46.964957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cd6e8-a69a-419f-947e-c4c84b00bbe7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:46.965266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cd6e8-a69a-419f-947e-c4c84b00bbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:46.965538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212cd6e8-a69a-419f-947e-c4c84b00bbe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:46.977277 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212cd6e8-a69a-419f-947e-c4c84b00bbe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:46.978772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212cd6e8-a69a-419f-947e-c4c84b00bbe7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:46.988700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea5e41-b439-4a28-a91b-a5b461b384fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:46.989639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea5e41-b439-4a28-a91b-a5b461b384fc None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:46.990138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea5e41-b439-4a28-a91b-a5b461b384fc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:46.990609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bea5e41-b439-4a28-a91b-a5b461b384fc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:46.991489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea5e41-b439-4a28-a91b-a5b461b384fc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:46.993892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bea5e41-b439-4a28-a91b-a5b461b384fc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-212cd6e8-a69a-419f-947e-c4c84b00bbe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:46.994879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:46.995894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:46.996794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:46.996794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:46.996794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:46.996794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 464/922] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:47.002988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cd6e8-a69a-419f-947e-c4c84b00bbe7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:47.038067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 459/923] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 21:57:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:57:47.104725 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:47.142682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212cd6e8-a69a-419f-947e-c4c84b00bbe7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:47.193863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4de1ee-91d6-4974-9756-ef82d011009f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:47.195530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4de1ee-91d6-4974-9756-ef82d011009f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:47.196021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4de1ee-91d6-4974-9756-ef82d011009f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:47.196522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4de1ee-91d6-4974-9756-ef82d011009f None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:47.197460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4de1ee-91d6-4974-9756-ef82d011009f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:47.197978 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4de1ee-91d6-4974-9756-ef82d011009f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:47.199614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4de1ee-91d6-4974-9756-ef82d011009f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:47.215246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 465/924] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:47.222304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea5e41-b439-4a28-a91b-a5b461b384fc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:47.278243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16583-1c26-4b68-8849-964a15890d94 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:47.278561 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16583-1c26-4b68-8849-964a15890d94 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:47.278794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f16583-1c26-4b68-8849-964a15890d94 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:47.281298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 466/925] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:57:47.286309 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78be8b5-6c01-4482-9953-0abc4ccf2e5c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:47.286829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78be8b5-6c01-4482-9953-0abc4ccf2e5c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:47.287310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78be8b5-6c01-4482-9953-0abc4ccf2e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:47.303145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:47.311453 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f78be8b5-6c01-4482-9953-0abc4ccf2e5c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:47.314890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f78be8b5-6c01-4482-9953-0abc4ccf2e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:47.337062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bea5e41-b439-4a28-a91b-a5b461b384fc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:47.375442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c41c3c-0329-4590-bf89-f4fea2e9fc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:47.376078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41c3c-0329-4590-bf89-f4fea2e9fc70 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:47.376352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41c3c-0329-4590-bf89-f4fea2e9fc70 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:47.376621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c41c3c-0329-4590-bf89-f4fea2e9fc70 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:47.385021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c41c3c-0329-4590-bf89-f4fea2e9fc70 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:47.392883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 467/926] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:47.393033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c41c3c-0329-4590-bf89-f4fea2e9fc70 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:47.393145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985a97c-c8aa-4478-b771-b232b7f4cdf2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:47.393145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985a97c-c8aa-4478-b771-b232b7f4cdf2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:47.393145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985a97c-c8aa-4478-b771-b232b7f4cdf2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:47.393145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 468/927] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:47.394353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e875f-c64a-4e6f-a534-8895ed8fb604 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:47.394638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e875f-c64a-4e6f-a534-8895ed8fb604 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:47.394911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e875f-c64a-4e6f-a534-8895ed8fb604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-48c41c3c-0329-4590-bf89-f4fea2e9fc70 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:47.406272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:47.407336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:47.408503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:47.408503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:47.408503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:47.408503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:47.408503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:47.408503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:47.408503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:47.408503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:47.408503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:47.408503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:47.410747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 460/928] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 22 21:57:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:47.416222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-487e875f-c64a-4e6f-a534-8895ed8fb604 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:47.416638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c41c3c-0329-4590-bf89-f4fea2e9fc70 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:47.418073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-487e875f-c64a-4e6f-a534-8895ed8fb604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:47.480741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:47.505233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 469/929] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:47.535799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c41c3c-0329-4590-bf89-f4fea2e9fc70 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:47.566815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:47.573236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be065187-b6fe-4b39-867e-59280b04e50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:47.574291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be065187-b6fe-4b39-867e-59280b04e50f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:47.574669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be065187-b6fe-4b39-867e-59280b04e50f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:47.575267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be065187-b6fe-4b39-867e-59280b04e50f None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:47.576418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be065187-b6fe-4b39-867e-59280b04e50f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:47.576777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be065187-b6fe-4b39-867e-59280b04e50f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:47.578301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be065187-b6fe-4b39-867e-59280b04e50f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:47.601600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 461/930] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 22 21:57:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:47.607652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e875f-c64a-4e6f-a534-8895ed8fb604 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:47.668776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be065187-b6fe-4b39-867e-59280b04e50f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:47.723967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e875f-c64a-4e6f-a534-8895ed8fb604 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:47.769507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5cde0-8da3-4369-ad8b-413dc5e26e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:47.770371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5cde0-8da3-4369-ad8b-413dc5e26e48 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:47.770798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5cde0-8da3-4369-ad8b-413dc5e26e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:47.771228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5cde0-8da3-4369-ad8b-413dc5e26e48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:47.785480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5cde0-8da3-4369-ad8b-413dc5e26e48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:47.787278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5cde0-8da3-4369-ad8b-413dc5e26e48 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:47.801609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be065187-b6fe-4b39-867e-59280b04e50f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-35e5cde0-8da3-4369-ad8b-413dc5e26e48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:47.806790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:47.807740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:47.808618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:47.808618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:47.808618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:47.812403 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 470/931] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:47.813267 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5cde0-8da3-4369-ad8b-413dc5e26e48 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:47.838668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5a938-b3f5-4531-9d06-4cdc461e1898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:47.839544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5a938-b3f5-4531-9d06-4cdc461e1898 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:47.839824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5a938-b3f5-4531-9d06-4cdc461e1898 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:47.840084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5a938-b3f5-4531-9d06-4cdc461e1898 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:47.840664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5a938-b3f5-4531-9d06-4cdc461e1898 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:47.842082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5a938-b3f5-4531-9d06-4cdc461e1898 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-dca5a938-b3f5-4531-9d06-4cdc461e1898 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:47.853144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:47.854169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:47.860611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:47.861416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:47.861490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 462/932] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 22 21:57:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:47.861490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5a938-b3f5-4531-9d06-4cdc461e1898 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:47.920349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:47.920801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5cde0-8da3-4369-ad8b-413dc5e26e48 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:47.957303 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ecb6e-e17f-47d2-ac6f-53e7975f6b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:47.957932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecb6e-e17f-47d2-ac6f-53e7975f6b62 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:47.958190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecb6e-e17f-47d2-ac6f-53e7975f6b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:47.958461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ecb6e-e17f-47d2-ac6f-53e7975f6b62 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:47.959041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ecb6e-e17f-47d2-ac6f-53e7975f6b62 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:47.960438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6ecb6e-e17f-47d2-ac6f-53e7975f6b62 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2b6ecb6e-e17f-47d2-ac6f-53e7975f6b62 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:47.969736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:47.970654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:47.971555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 471/933] 10.222.0.25 () {56 vars in 1132 bytes} [Thu Sep 22 21:57:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:47.972979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ecb6e-e17f-47d2-ac6f-53e7975f6b62 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:47.974828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5a938-b3f5-4531-9d06-4cdc461e1898 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:48.010496 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e496b0b-6787-46da-aa77-e9e66c0b725c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:48.011121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e496b0b-6787-46da-aa77-e9e66c0b725c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:48.011395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e496b0b-6787-46da-aa77-e9e66c0b725c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:48.011666 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e496b0b-6787-46da-aa77-e9e66c0b725c None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:48.012295 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e496b0b-6787-46da-aa77-e9e66c0b725c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:48.012623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e496b0b-6787-46da-aa77-e9e66c0b725c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:48.013998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e496b0b-6787-46da-aa77-e9e66c0b725c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:48.031537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 463/934] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:48.031537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e496b0b-6787-46da-aa77-e9e66c0b725c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:48.087999 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ecb6e-e17f-47d2-ac6f-53e7975f6b62 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:48.089915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:48.125893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35a5d5-0b54-4dce-a7f4-151b5f3fac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:48.126859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35a5d5-0b54-4dce-a7f4-151b5f3fac3e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:48.127226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35a5d5-0b54-4dce-a7f4-151b5f3fac3e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:48.127468 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35a5d5-0b54-4dce-a7f4-151b5f3fac3e None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:48.128020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35a5d5-0b54-4dce-a7f4-151b5f3fac3e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:48.128318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35a5d5-0b54-4dce-a7f4-151b5f3fac3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:48.129700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35a5d5-0b54-4dce-a7f4-151b5f3fac3e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:48.135041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 472/935] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 22 21:57:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:48.135806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35a5d5-0b54-4dce-a7f4-151b5f3fac3e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:48.150196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e496b0b-6787-46da-aa77-e9e66c0b725c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:48.166112 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:48.186994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcca07-e5df-4e8e-b20c-489823de0e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:48.187592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcca07-e5df-4e8e-b20c-489823de0e35 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:48.187868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcca07-e5df-4e8e-b20c-489823de0e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:48.188138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bcca07-e5df-4e8e-b20c-489823de0e35 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:48.188901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcca07-e5df-4e8e-b20c-489823de0e35 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:48.189076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcca07-e5df-4e8e-b20c-489823de0e35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:48.190490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bcca07-e5df-4e8e-b20c-489823de0e35 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:48.196054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 464/936] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 22 21:57:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:48.196822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcca07-e5df-4e8e-b20c-489823de0e35 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:48.245390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35a5d5-0b54-4dce-a7f4-151b5f3fac3e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:48.281009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888eae9-e6fe-4fd9-a092-db72b6ef382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:48.281664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888eae9-e6fe-4fd9-a092-db72b6ef382b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:48.281950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888eae9-e6fe-4fd9-a092-db72b6ef382b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:48.282219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b888eae9-e6fe-4fd9-a092-db72b6ef382b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:48.289644 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888eae9-e6fe-4fd9-a092-db72b6ef382b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:48.291065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b888eae9-e6fe-4fd9-a092-db72b6ef382b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:48.299665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bcca07-e5df-4e8e-b20c-489823de0e35 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b888eae9-e6fe-4fd9-a092-db72b6ef382b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:48.302995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:48.303854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:48.305692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:48.305692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:48.305692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:48.305692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:48.305692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:48.305692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:48.305692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:48.305692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:48.305692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 473/937] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:48.306205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888eae9-e6fe-4fd9-a092-db72b6ef382b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:48.334553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:48.340220 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b481b63-9ffb-4231-beef-52c92f68ee1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:48.340817 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b481b63-9ffb-4231-beef-52c92f68ee1c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:48.341089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b481b63-9ffb-4231-beef-52c92f68ee1c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:48.341380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b481b63-9ffb-4231-beef-52c92f68ee1c None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:48.341998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b481b63-9ffb-4231-beef-52c92f68ee1c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:48.342296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b481b63-9ffb-4231-beef-52c92f68ee1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:48.343651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b481b63-9ffb-4231-beef-52c92f68ee1c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:48.348598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 465/938] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 22 21:57:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:48.349403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b481b63-9ffb-4231-beef-52c92f68ee1c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:48.398740 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:48.421212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b888eae9-e6fe-4fd9-a092-db72b6ef382b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:48.457181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459f7e-a90f-4a10-9e72-effd83f69921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:48.457787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459f7e-a90f-4a10-9e72-effd83f69921 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:48.458063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459f7e-a90f-4a10-9e72-effd83f69921 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:48.458332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1459f7e-a90f-4a10-9e72-effd83f69921 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:48.458946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1459f7e-a90f-4a10-9e72-effd83f69921 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:48.459246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1459f7e-a90f-4a10-9e72-effd83f69921 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:48.460386 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b481b63-9ffb-4231-beef-52c92f68ee1c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:48.460699 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1459f7e-a90f-4a10-9e72-effd83f69921 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:48.466696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 474/939] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 22 21:57:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:48.471831 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459f7e-a90f-4a10-9e72-effd83f69921 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:48.496906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b0d1e-9fb6-43c3-b001-596bb952d68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:48.497556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0d1e-9fb6-43c3-b001-596bb952d68c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:48.497834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0d1e-9fb6-43c3-b001-596bb952d68c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:48.498103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345b0d1e-9fb6-43c3-b001-596bb952d68c None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:48.498774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b0d1e-9fb6-43c3-b001-596bb952d68c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:48.499078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b0d1e-9fb6-43c3-b001-596bb952d68c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:48.500491 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345b0d1e-9fb6-43c3-b001-596bb952d68c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:48.513151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 466/940] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:48.513759 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b0d1e-9fb6-43c3-b001-596bb952d68c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:48.567808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:48.579891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1459f7e-a90f-4a10-9e72-effd83f69921 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:48.611748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345b0d1e-9fb6-43c3-b001-596bb952d68c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:48.613183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:48.614017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12acb84-5270-499e-90cc-87b0b9a43d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:48.614621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12acb84-5270-499e-90cc-87b0b9a43d4d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:48.614899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12acb84-5270-499e-90cc-87b0b9a43d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:48.615171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12acb84-5270-499e-90cc-87b0b9a43d4d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:48.615874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12acb84-5270-499e-90cc-87b0b9a43d4d None admin] RBAC: Adding query filter params (interface=public, service_id=bb979e6ccb8a408d83ec3fe282c31842, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:48.616206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12acb84-5270-499e-90cc-87b0b9a43d4d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:48.618089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12acb84-5270-499e-90cc-87b0b9a43d4d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:48.623628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 475/941] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 22 21:57:48 2022] GET /identity/v3/endpoints?service_id=bb979e6ccb8a408d83ec3fe282c31842&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:48.624379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12acb84-5270-499e-90cc-87b0b9a43d4d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:48.652588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4848-22ac-4ac2-bd90-5a8aa1c77efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:48.653213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4848-22ac-4ac2-bd90-5a8aa1c77efb None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:48.653490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4848-22ac-4ac2-bd90-5a8aa1c77efb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:48.653756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb4848-22ac-4ac2-bd90-5a8aa1c77efb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:48.654409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb4848-22ac-4ac2-bd90-5a8aa1c77efb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:48.655743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb4848-22ac-4ac2-bd90-5a8aa1c77efb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:48.677202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 467/942] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 21:57:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:57:48.678017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4848-22ac-4ac2-bd90-5a8aa1c77efb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:48.745142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12acb84-5270-499e-90cc-87b0b9a43d4d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:48.790875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e8d1e4-e684-4764-8568-77ac29d49f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:48.791541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8d1e4-e684-4764-8568-77ac29d49f16 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:48.791818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8d1e4-e684-4764-8568-77ac29d49f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:48.792085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e8d1e4-e684-4764-8568-77ac29d49f16 None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/bf3dde2bfc4240a8af6425778c5eddaf/roles/f1c696334f084cccb674b2528a9fa65c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:48.794460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8d1e4-e684-4764-8568-77ac29d49f16 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=bf3dde2bfc4240a8af6425778c5eddaf, role_id=f1c696334f084cccb674b2528a9fa65c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:48.795910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e8d1e4-e684-4764-8568-77ac29d49f16 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:48.817215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 476/943] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:57:48 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/bf3dde2bfc4240a8af6425778c5eddaf/roles/f1c696334f084cccb674b2528a9fa65c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:57:48.837563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb4848-22ac-4ac2-bd90-5a8aa1c77efb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:48.882562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5820f40-62b5-461d-bb2f-ef3842ce8b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:48.883581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5820f40-62b5-461d-bb2f-ef3842ce8b3a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:48.883959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5820f40-62b5-461d-bb2f-ef3842ce8b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:48.884329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5820f40-62b5-461d-bb2f-ef3842ce8b3a None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:48.885059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5820f40-62b5-461d-bb2f-ef3842ce8b3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:48.887830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5820f40-62b5-461d-bb2f-ef3842ce8b3a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:48.895027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 468/944] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 22 21:57:48 2022] GET /identity/v3/services => generated 1497 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 21:57:49.360308 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839ac68-f481-4339-baa5-b3cbaf192a4f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:49.360868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839ac68-f481-4339-baa5-b3cbaf192a4f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:49.360868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6839ac68-f481-4339-baa5-b3cbaf192a4f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:49.362481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 477/945] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:49.368392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07922ac-3e42-4df2-8fcd-517a24fe96e6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:49.368821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07922ac-3e42-4df2-8fcd-517a24fe96e6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:49.368998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07922ac-3e42-4df2-8fcd-517a24fe96e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:49.388223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e07922ac-3e42-4df2-8fcd-517a24fe96e6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:49.389460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e07922ac-3e42-4df2-8fcd-517a24fe96e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:49.407216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:49.466009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 469/946] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:49.734051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839ac68-f481-4339-baa5-b3cbaf192a4f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:49.747847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:49.849200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6839ac68-f481-4339-baa5-b3cbaf192a4f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:49.884206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e78cb0-9238-4f10-b436-aa6df0e7b5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:49.884932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e78cb0-9238-4f10-b436-aa6df0e7b5f9 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:49.885330 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e78cb0-9238-4f10-b436-aa6df0e7b5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:49.885694 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e78cb0-9238-4f10-b436-aa6df0e7b5f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:49.886457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e78cb0-9238-4f10-b436-aa6df0e7b5f9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:49.887958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e78cb0-9238-4f10-b436-aa6df0e7b5f9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:49.890650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: WARNING keystone.server.flask.common [None req-f9e78cb0-9238-4f10-b436-aa6df0e7b5f9 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 22 21:57:49.910446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.sql.core [None req-f9e78cb0-9238-4f10-b436-aa6df0e7b5f9 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 21:57:49.910446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 21:57:49.910446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [parameters: {'id': 'bb0299ee2fed4ad6b1290f3ca8ccabf0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 21:57:49.910446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81629) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f9e78cb0-9238-4f10-b436-aa6df0e7b5f9 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb0299ee2fed4ad6b1290f3ca8ccabf0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb0299ee2fed4ad6b1290f3ca8ccabf0. Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 21:57:49.927628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 22 21:57:49.928613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application yield resource Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application next(self.gen) Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise self.value Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application session.commit() Sep 22 21:57:49.929918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.session.flush() Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application compat.raise_( Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 21:57:49.931234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 21:57:49.933088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application conn.query(q) Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application result.read() Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 21:57:49.934832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [parameters: {'id': 'bb0299ee2fed4ad6b1290f3ca8ccabf0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:49.937752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:49.943384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:49.946071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:49.946071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 22 21:57:49.946071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 22 21:57:49.946071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:49.946071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:49.946071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 22 21:57:49.946071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 22 21:57:49.946071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb0299ee2fed4ad6b1290f3ca8ccabf0. Sep 22 21:57:49.946071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:49.946071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 478/947] 10.222.0.25 () {60 vars in 1140 bytes} [Thu Sep 22 21:57:49 2022] POST /identity/v3/projects => generated 262 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:49.949137 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07922ac-3e42-4df2-8fcd-517a24fe96e6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:50.055534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07922ac-3e42-4df2-8fcd-517a24fe96e6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:50.092525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941d45f-ee31-4c39-9757-164d0952f31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:50.093233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941d45f-ee31-4c39-9757-164d0952f31a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:50.093511 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941d45f-ee31-4c39-9757-164d0952f31a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:50.093782 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941d45f-ee31-4c39-9757-164d0952f31a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:50.101616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941d45f-ee31-4c39-9757-164d0952f31a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:50.103229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941d45f-ee31-4c39-9757-164d0952f31a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-e941d45f-ee31-4c39-9757-164d0952f31a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:50.114861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:50.115767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:50.117006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:50.117006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:50.117006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:50.117006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:50.117006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:50.117006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:50.117006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:50.117006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:50.117006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 470/948] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:50.124158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e78cb0-9238-4f10-b436-aa6df0e7b5f9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:50.192025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:50.228306 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e78cb0-9238-4f10-b436-aa6df0e7b5f9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:50.263240 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a7fd2-9804-4e42-b856-b825610730c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:50.264237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a7fd2-9804-4e42-b856-b825610730c0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:50.264707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a7fd2-9804-4e42-b856-b825610730c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:50.265213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a7fd2-9804-4e42-b856-b825610730c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:50.265917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a7fd2-9804-4e42-b856-b825610730c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:50.266218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a7fd2-9804-4e42-b856-b825610730c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:50.268382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853a7fd2-9804-4e42-b856-b825610730c0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:50.289553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 479/949] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:57:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:50.990382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0546f2-3b37-41a0-99c5-e230a106c95e None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:50.990908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0546f2-3b37-41a0-99c5-e230a106c95e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:50.990908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0546f2-3b37-41a0-99c5-e230a106c95e None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:50.992806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 471/950] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:50.998693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9af2d-57ca-457b-a3c5-3f103dc2e564 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:50.999206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9af2d-57ca-457b-a3c5-3f103dc2e564 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:50.999295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d9af2d-57ca-457b-a3c5-3f103dc2e564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:51.018883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-54d9af2d-57ca-457b-a3c5-3f103dc2e564 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:51.020409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-54d9af2d-57ca-457b-a3c5-3f103dc2e564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:51.097755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 480/951] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:51.107237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b9574-47f4-4256-9a1c-c5a7674b10ae None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:51.107892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b9574-47f4-4256-9a1c-c5a7674b10ae None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:51.108368 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b9574-47f4-4256-9a1c-c5a7674b10ae None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:51.109923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 472/952] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:51.115870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb652f1-d3f8-45c5-9e10-8a2d920fd29a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:51.116494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb652f1-d3f8-45c5-9e10-8a2d920fd29a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:51.117054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb652f1-d3f8-45c5-9e10-8a2d920fd29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:51.135886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9eb652f1-d3f8-45c5-9e10-8a2d920fd29a None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:51.137427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9eb652f1-d3f8-45c5-9e10-8a2d920fd29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:51.197497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:51.204674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5147297-343b-45d4-9c25-1c5838e04693 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:51.205358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5147297-343b-45d4-9c25-1c5838e04693 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:51.205899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5147297-343b-45d4-9c25-1c5838e04693 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:51.210777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 481/954] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:51.211098 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 473/954] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:51.215200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118150d-69e7-463e-82e3-9305f3319205 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:51.215482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118150d-69e7-463e-82e3-9305f3319205 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:51.215755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9118150d-69e7-463e-82e3-9305f3319205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:51.235171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9118150d-69e7-463e-82e3-9305f3319205 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:51.236412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9118150d-69e7-463e-82e3-9305f3319205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:51.314997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 482/955] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:51.371293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5147297-343b-45d4-9c25-1c5838e04693 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:51.387537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:51.477495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5147297-343b-45d4-9c25-1c5838e04693 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:51.503245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118150d-69e7-463e-82e3-9305f3319205 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:51.514837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d031759-2e68-44f5-af7f-259553ffd17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:51.515773 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d031759-2e68-44f5-af7f-259553ffd17d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:51.516305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d031759-2e68-44f5-af7f-259553ffd17d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:51.516776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d031759-2e68-44f5-af7f-259553ffd17d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:51.517583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d031759-2e68-44f5-af7f-259553ffd17d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:51.519146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d031759-2e68-44f5-af7f-259553ffd17d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-5d031759-2e68-44f5-af7f-259553ffd17d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:51.528479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:51.529612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 21:57:51.530502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:51.533246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 474/956] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:51.539311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d031759-2e68-44f5-af7f-259553ffd17d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:51.629158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9118150d-69e7-463e-82e3-9305f3319205 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:51.643311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:51.650925 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:51.671052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d031759-2e68-44f5-af7f-259553ffd17d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:51.679800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67e058-05ae-4e59-98f2-61ec2bf2ad93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:51.680444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67e058-05ae-4e59-98f2-61ec2bf2ad93 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:51.680718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67e058-05ae-4e59-98f2-61ec2bf2ad93 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:51.680985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67e058-05ae-4e59-98f2-61ec2bf2ad93 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:51.692427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a67e058-05ae-4e59-98f2-61ec2bf2ad93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2a67e058-05ae-4e59-98f2-61ec2bf2ad93 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:57:51.698033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:51.698981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:57:51.699853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:51.704048 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 483/957] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 22 21:57:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:51.704867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67e058-05ae-4e59-98f2-61ec2bf2ad93 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:51.709735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c775d5c-fd1b-4656-b8ad-66f6d91ff491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:51.711020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c775d5c-fd1b-4656-b8ad-66f6d91ff491 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:51.711337 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c775d5c-fd1b-4656-b8ad-66f6d91ff491 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:51.711598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c775d5c-fd1b-4656-b8ad-66f6d91ff491 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:51.712214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c775d5c-fd1b-4656-b8ad-66f6d91ff491 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:51.712507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c775d5c-fd1b-4656-b8ad-66f6d91ff491 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:51.714143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c775d5c-fd1b-4656-b8ad-66f6d91ff491 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:51.719685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 475/958] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:57:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:51.720498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c775d5c-fd1b-4656-b8ad-66f6d91ff491 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:51.817749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a67e058-05ae-4e59-98f2-61ec2bf2ad93 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:51.819020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c775d5c-fd1b-4656-b8ad-66f6d91ff491 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:51.852888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44468f2d-a04e-4fc7-913b-e9fe1d95ce25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:51.853522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44468f2d-a04e-4fc7-913b-e9fe1d95ce25 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:51.853798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44468f2d-a04e-4fc7-913b-e9fe1d95ce25 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:51.854065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44468f2d-a04e-4fc7-913b-e9fe1d95ce25 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:51.854707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44468f2d-a04e-4fc7-913b-e9fe1d95ce25 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:51.855004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44468f2d-a04e-4fc7-913b-e9fe1d95ce25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:51.855816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922660a-f9b6-486e-a824-715bf0b94fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:51.856429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922660a-f9b6-486e-a824-715bf0b94fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:51.856604 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44468f2d-a04e-4fc7-913b-e9fe1d95ce25 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:51.856700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922660a-f9b6-486e-a824-715bf0b94fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:51.856954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2922660a-f9b6-486e-a824-715bf0b94fc9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:51.857555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922660a-f9b6-486e-a824-715bf0b94fc9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:51.858887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2922660a-f9b6-486e-a824-715bf0b94fc9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:51.864648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 476/959] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 22 21:57:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:51.865598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44468f2d-a04e-4fc7-913b-e9fe1d95ce25 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2922660a-f9b6-486e-a824-715bf0b94fc9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:51.867760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:51.868715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:51.869608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 484/960] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 22 21:57:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:51.872972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:51.875038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922660a-f9b6-486e-a824-715bf0b94fc9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:51.919137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:51.975777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44468f2d-a04e-4fc7-913b-e9fe1d95ce25 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:51.985724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2922660a-f9b6-486e-a824-715bf0b94fc9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:52.009680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be28f12-327c-4033-a713-fb97bc79527c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:52.010337 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be28f12-327c-4033-a713-fb97bc79527c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:52.010543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be28f12-327c-4033-a713-fb97bc79527c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:52.010809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be28f12-327c-4033-a713-fb97bc79527c None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:52.011418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be28f12-327c-4033-a713-fb97bc79527c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:52.011713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be28f12-327c-4033-a713-fb97bc79527c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:52.013064 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be28f12-327c-4033-a713-fb97bc79527c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:52.017573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 477/961] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:57:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:52.018333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be28f12-327c-4033-a713-fb97bc79527c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:52.035281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b672f72-175e-4462-8cab-14352776871e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:52.036805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b672f72-175e-4462-8cab-14352776871e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:52.037332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b672f72-175e-4462-8cab-14352776871e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:52.037675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b672f72-175e-4462-8cab-14352776871e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:52.053204 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b672f72-175e-4462-8cab-14352776871e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:52.055126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b672f72-175e-4462-8cab-14352776871e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-9b672f72-175e-4462-8cab-14352776871e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:52.073255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:52.074500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:52.075630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:52.075630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:52.075630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:52.076567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 485/962] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:52.083294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b672f72-175e-4462-8cab-14352776871e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:52.126659 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:52.176842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be28f12-327c-4033-a713-fb97bc79527c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:52.199032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:52.211099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b672f72-175e-4462-8cab-14352776871e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:52.226572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06fd40-f388-40d3-b4b6-8200ebd4f263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:52.227571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06fd40-f388-40d3-b4b6-8200ebd4f263 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:52.227856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06fd40-f388-40d3-b4b6-8200ebd4f263 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:52.228125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06fd40-f388-40d3-b4b6-8200ebd4f263 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:52.228755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06fd40-f388-40d3-b4b6-8200ebd4f263 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:52.229055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06fd40-f388-40d3-b4b6-8200ebd4f263 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:52.230931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c06fd40-f388-40d3-b4b6-8200ebd4f263 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:52.236045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 478/963] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 22 21:57:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:52.242814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06fd40-f388-40d3-b4b6-8200ebd4f263 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:52.269969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97b460-2a19-4e02-9db2-e05d9fc2606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:52.270893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97b460-2a19-4e02-9db2-e05d9fc2606d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:52.271616 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97b460-2a19-4e02-9db2-e05d9fc2606d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:52.271992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97b460-2a19-4e02-9db2-e05d9fc2606d None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:52.273507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97b460-2a19-4e02-9db2-e05d9fc2606d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:52.273962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97b460-2a19-4e02-9db2-e05d9fc2606d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:52.275407 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97b460-2a19-4e02-9db2-e05d9fc2606d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:52.292073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 486/964] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:52.297659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97b460-2a19-4e02-9db2-e05d9fc2606d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:52.348163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06fd40-f388-40d3-b4b6-8200ebd4f263 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:52.381999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:52.386179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5921a-aabd-4e38-9f6f-c037092f0567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:52.386962 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5921a-aabd-4e38-9f6f-c037092f0567 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:52.387304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5921a-aabd-4e38-9f6f-c037092f0567 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:52.387631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5921a-aabd-4e38-9f6f-c037092f0567 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:52.388326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5921a-aabd-4e38-9f6f-c037092f0567 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:52.388697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5921a-aabd-4e38-9f6f-c037092f0567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:52.390421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5921a-aabd-4e38-9f6f-c037092f0567 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:52.399757 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97b460-2a19-4e02-9db2-e05d9fc2606d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:52.401180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 479/965] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 22 21:57:52 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:52.402229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5921a-aabd-4e38-9f6f-c037092f0567 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:52.437186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d1fd8-2530-452e-947e-e49b621edc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:52.437818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d1fd8-2530-452e-947e-e49b621edc8f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:52.438093 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d1fd8-2530-452e-947e-e49b621edc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:52.438361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d1fd8-2530-452e-947e-e49b621edc8f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:52.447019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d1fd8-2530-452e-947e-e49b621edc8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:52.448549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d1fd8-2530-452e-947e-e49b621edc8f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-c56d1fd8-2530-452e-947e-e49b621edc8f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:52.462547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:52.463687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:52.464917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:52.464917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:52.464917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:52.464917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:52.464917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:52.464917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:52.464917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:52.464917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:52.464917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 22 21:57:52.464917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:52.468994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 487/966] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 22 21:57:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:52.470052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d1fd8-2530-452e-947e-e49b621edc8f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:52.487141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:52.516168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5921a-aabd-4e38-9f6f-c037092f0567 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:52.553929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06333a-c026-4ed8-96b6-4f25ee8a0ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:52.554810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06333a-c026-4ed8-96b6-4f25ee8a0ced None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:52.555229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06333a-c026-4ed8-96b6-4f25ee8a0ced None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:52.555654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06333a-c026-4ed8-96b6-4f25ee8a0ced None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:52.556480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06333a-c026-4ed8-96b6-4f25ee8a0ced None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:52.558183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06333a-c026-4ed8-96b6-4f25ee8a0ced None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:52.577770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d1fd8-2530-452e-947e-e49b621edc8f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:52.584207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 480/967] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 21:57:52 2022] POST /identity/v3/endpoints => generated 349 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:57:52.587263 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960927-3f1e-49a8-9b5e-ffcaa92120ae None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:52.587797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960927-3f1e-49a8-9b5e-ffcaa92120ae None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:52.588217 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54960927-3f1e-49a8-9b5e-ffcaa92120ae None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:52.590835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 481/968] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:52.598268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c1f5a-e512-4562-9f9f-468425f519fd None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:52.598646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c1f5a-e512-4562-9f9f-468425f519fd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:52.598818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c1f5a-e512-4562-9f9f-468425f519fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:52.625230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c30c1f5a-e512-4562-9f9f-468425f519fd None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:52.626479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c30c1f5a-e512-4562-9f9f-468425f519fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:52.629758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c656212-3e39-487d-91d3-41c37a40326a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:52.630443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c656212-3e39-487d-91d3-41c37a40326a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:52.630724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c656212-3e39-487d-91d3-41c37a40326a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:52.630994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c656212-3e39-487d-91d3-41c37a40326a None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:52.631733 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c656212-3e39-487d-91d3-41c37a40326a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:52.632808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c656212-3e39-487d-91d3-41c37a40326a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:52.634140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c656212-3e39-487d-91d3-41c37a40326a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:52.656189 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 488/969] 10.222.0.25 () {56 vars in 1149 bytes} [Thu Sep 22 21:57:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:52.662559 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c656212-3e39-487d-91d3-41c37a40326a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:52.700067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:52.737134 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 482/970] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:52.746916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:52.786907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c656212-3e39-487d-91d3-41c37a40326a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:52.823682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98493ca-e7ed-459d-915e-a844baddd341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:52.824297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98493ca-e7ed-459d-915e-a844baddd341 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:52.824574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98493ca-e7ed-459d-915e-a844baddd341 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:52.824852 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98493ca-e7ed-459d-915e-a844baddd341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:52.894943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98493ca-e7ed-459d-915e-a844baddd341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:52.896425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98493ca-e7ed-459d-915e-a844baddd341 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:52.968027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:53.008255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 489/971] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:57:52 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 21:57:53.013822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c1f5a-e512-4562-9f9f-468425f519fd None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:53.014283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98493ca-e7ed-459d-915e-a844baddd341 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:53.122889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98493ca-e7ed-459d-915e-a844baddd341 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:53.127443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c1f5a-e512-4562-9f9f-468425f519fd None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:53.160688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c47ac-479f-401b-b1d7-01f19fba7174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:53.161327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c47ac-479f-401b-b1d7-01f19fba7174 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:53.161606 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c47ac-479f-401b-b1d7-01f19fba7174 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:53.161879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248c47ac-479f-401b-b1d7-01f19fba7174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:53.173558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1f7b8-2706-4f73-abaa-2565e2108613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:53.173741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c47ac-479f-401b-b1d7-01f19fba7174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:53.174360 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1f7b8-2706-4f73-abaa-2565e2108613 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:53.174744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1f7b8-2706-4f73-abaa-2565e2108613 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:53.174993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248c47ac-479f-401b-b1d7-01f19fba7174 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:53.175193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f1f7b8-2706-4f73-abaa-2565e2108613 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:53.175951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1f7b8-2706-4f73-abaa-2565e2108613 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:53.178172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f1f7b8-2706-4f73-abaa-2565e2108613 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:53.181043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: WARNING keystone.server.flask.common [None req-79f1f7b8-2706-4f73-abaa-2565e2108613 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-248c47ac-479f-401b-b1d7-01f19fba7174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:57:53.190060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:53.191020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:57:53.191921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:57:53.191921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:57:53.191921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:53.191921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 490/972] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:53.198451 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c47ac-479f-401b-b1d7-01f19fba7174 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:53.203630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.sql.core [None req-79f1f7b8-2706-4f73-abaa-2565e2108613 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 21:57:53.203630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 21:57:53.203630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [parameters: {'user_id': 'a09a70b0d3674b4d9ab38cf4ebbd00c5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 21:57:53.203630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81628) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-79f1f7b8-2706-4f73-abaa-2565e2108613 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application conn.query(q) Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result.read() Sep 22 21:57:53.214709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 21:57:53.215608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application next(self.gen) Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application yield resource Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application next(self.gen) Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.session.rollback() Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.force_reraise() Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise self.value Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application session.commit() Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 22 21:57:53.216527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._prepare_impl() Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.session.flush() Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._flush(objects) Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application compat.raise_( Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application flush_context.execute() Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rec.execute(self) Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 22 21:57:53.217549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 22 21:57:53.218495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result = self._query(query) Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application conn.query(q) Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application result.read() Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 22 21:57:53.221358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a09a70b0d3674b4d9ab38cf4ebbd00c5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:53.223475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 22 21:57:53.226662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 22 21:57:53.229828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 22 21:57:53.229828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 22 21:57:53.229828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 22 21:57:53.229828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 22 21:57:53.229828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 22 21:57:53.229828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:53.229828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 483/973] 10.222.0.25 () {60 vars in 1131 bytes} [Thu Sep 22 21:57:53 2022] POST /identity/v3/users => generated 160 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:53.229828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1f7b8-2706-4f73-abaa-2565e2108613 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:53.233180 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:53.279012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:53.340735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248c47ac-479f-401b-b1d7-01f19fba7174 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:53.343336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f1f7b8-2706-4f73-abaa-2565e2108613 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:53.380655 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a6877-9001-4aa1-9fc1-e105167d9bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:53.381336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a6877-9001-4aa1-9fc1-e105167d9bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:53.381700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a6877-9001-4aa1-9fc1-e105167d9bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:53.382149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a6877-9001-4aa1-9fc1-e105167d9bc4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:53.385026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e6e20-799a-4786-b290-ad360be7fee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:53.386501 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e6e20-799a-4786-b290-ad360be7fee4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:53.387009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e6e20-799a-4786-b290-ad360be7fee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:53.387481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53e6e20-799a-4786-b290-ad360be7fee4 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:53.388318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e6e20-799a-4786-b290-ad360be7fee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:53.388829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e6e20-799a-4786-b290-ad360be7fee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:53.390444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a6877-9001-4aa1-9fc1-e105167d9bc4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:53.391118 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53e6e20-799a-4786-b290-ad360be7fee4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:53.392138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a6877-9001-4aa1-9fc1-e105167d9bc4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-4b3a6877-9001-4aa1-9fc1-e105167d9bc4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:57:53.408702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:53.409734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:53.410689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:53.410689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:53.410689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:57:53.410689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:57:53.410689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:57:53.410689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:57:53.410689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:57:53.410689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:57:53.410689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 22 21:57:53.410689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:53.411229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 491/975] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:53.411303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 484/975] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:57:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:53.412043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e6e20-799a-4786-b290-ad360be7fee4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:53.413884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a6877-9001-4aa1-9fc1-e105167d9bc4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:53.511320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:53.524970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a6877-9001-4aa1-9fc1-e105167d9bc4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:53.526536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:53.539365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53e6e20-799a-4786-b290-ad360be7fee4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:53.562703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c07e6-c153-42e4-b2dc-158bf4d856f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:53.563312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c07e6-c153-42e4-b2dc-158bf4d856f4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:53.563591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c07e6-c153-42e4-b2dc-158bf4d856f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:53.563866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c07e6-c153-42e4-b2dc-158bf4d856f4 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:53.564561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c07e6-c153-42e4-b2dc-158bf4d856f4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:53.564865 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c07e6-c153-42e4-b2dc-158bf4d856f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:53.566321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c07e6-c153-42e4-b2dc-158bf4d856f4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:53.581544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40e5f4-04f0-4095-84c9-b123a48faa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:53.582456 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e5f4-04f0-4095-84c9-b123a48faa69 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:53.582955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e5f4-04f0-4095-84c9-b123a48faa69 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:53.583484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e5f4-04f0-4095-84c9-b123a48faa69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:53.591860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40e5f4-04f0-4095-84c9-b123a48faa69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:53.594490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40e5f4-04f0-4095-84c9-b123a48faa69 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-5f40e5f4-04f0-4095-84c9-b123a48faa69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:57:53.612171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:53.614050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:53.615924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:57:53.615924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:57:53.615924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:57:53.615924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:57:53.615924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:57:53.615924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:57:53.615924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:57:53.615924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:53.616850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 485/976] 10.222.0.25 () {56 vars in 1101 bytes} [Thu Sep 22 21:57:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:53.624652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 492/977] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:57:53.626383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c07e6-c153-42e4-b2dc-158bf4d856f4 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:53.746981 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c07e6-c153-42e4-b2dc-158bf4d856f4 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:53.787372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02586676-9909-4e6c-808d-d3983e0ff2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:53.788915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02586676-9909-4e6c-808d-d3983e0ff2cb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:53.789506 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02586676-9909-4e6c-808d-d3983e0ff2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:53.789983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02586676-9909-4e6c-808d-d3983e0ff2cb None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:53.790961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02586676-9909-4e6c-808d-d3983e0ff2cb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:53.791502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02586676-9909-4e6c-808d-d3983e0ff2cb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:53.793194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02586676-9909-4e6c-808d-d3983e0ff2cb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:53.812017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 486/978] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:57:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:53.813659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40e5f4-04f0-4095-84c9-b123a48faa69 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:53.926593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40e5f4-04f0-4095-84c9-b123a48faa69 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:53.962208 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c17ea9d-a0c2-48cf-8f0f-7a552885f4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:53.962815 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17ea9d-a0c2-48cf-8f0f-7a552885f4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:53.963094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17ea9d-a0c2-48cf-8f0f-7a552885f4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:53.963368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c17ea9d-a0c2-48cf-8f0f-7a552885f4c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:53.964345 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17ea9d-a0c2-48cf-8f0f-7a552885f4c8 None admin] RBAC: Adding query filter params (role.id=f1c696334f084cccb674b2528a9fa65c, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=bf3dde2bfc4240a8af6425778c5eddaf) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:53.964653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17ea9d-a0c2-48cf-8f0f-7a552885f4c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:53.966096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c17ea9d-a0c2-48cf-8f0f-7a552885f4c8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:53.976889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 493/979] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:57:53 2022] GET /identity/v3/role_assignments?user.id=bf3dde2bfc4240a8af6425778c5eddaf&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=f1c696334f084cccb674b2528a9fa65c => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:54.288320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124f19f-679a-4431-b186-df71e2e4822f None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:54.288866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124f19f-679a-4431-b186-df71e2e4822f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:54.288866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124f19f-679a-4431-b186-df71e2e4822f None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:54.290348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 487/980] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:54.295459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb013dbe-9e43-4d32-96bd-4870a9cc7838 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:54.295844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb013dbe-9e43-4d32-96bd-4870a9cc7838 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:54.296040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb013dbe-9e43-4d32-96bd-4870a9cc7838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:54.302229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:54.315160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fb013dbe-9e43-4d32-96bd-4870a9cc7838 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:54.316259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fb013dbe-9e43-4d32-96bd-4870a9cc7838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:54.389135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 494/981] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:54.655419 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124f19f-679a-4431-b186-df71e2e4822f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:54.677945 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:54.762957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124f19f-679a-4431-b186-df71e2e4822f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:54.796653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e53eae1-52d8-4e6f-8811-01ec175cf7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:54.797277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53eae1-52d8-4e6f-8811-01ec175cf7ae None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:54.797564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53eae1-52d8-4e6f-8811-01ec175cf7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:54.797834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e53eae1-52d8-4e6f-8811-01ec175cf7ae None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:54.798497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53eae1-52d8-4e6f-8811-01ec175cf7ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:54.799859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e53eae1-52d8-4e6f-8811-01ec175cf7ae None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:54.817701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 488/982] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:57:54 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:57:54.952545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6933268-da5f-4780-b81e-aa40b1b33dd1 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:54.952754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6933268-da5f-4780-b81e-aa40b1b33dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:54.952989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6933268-da5f-4780-b81e-aa40b1b33dd1 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:54.954598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 495/983] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:54.960029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5e156-21d9-4ed9-ad12-ef03283d3576 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:54.960347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5e156-21d9-4ed9-ad12-ef03283d3576 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:54.960630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5e156-21d9-4ed9-ad12-ef03283d3576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:54.981309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-eae5e156-21d9-4ed9-ad12-ef03283d3576 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:54.982867 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-eae5e156-21d9-4ed9-ad12-ef03283d3576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:55.063532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 489/984] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:54 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:55.328156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6933268-da5f-4780-b81e-aa40b1b33dd1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:55.425991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6933268-da5f-4780-b81e-aa40b1b33dd1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:55.454191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:55.458745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860b712-567c-462d-87ae-9bdbb71d35c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:55.459366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860b712-567c-462d-87ae-9bdbb71d35c4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:55.459662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860b712-567c-462d-87ae-9bdbb71d35c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:55.460598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8860b712-567c-462d-87ae-9bdbb71d35c4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:55.461402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860b712-567c-462d-87ae-9bdbb71d35c4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:55.462927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8860b712-567c-462d-87ae-9bdbb71d35c4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-8860b712-567c-462d-87ae-9bdbb71d35c4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:55.471890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:55.472992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:55.474289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:55.477432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 496/985] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 22 21:57:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:55.478878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5e156-21d9-4ed9-ad12-ef03283d3576 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:55.487382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:55.578671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5e156-21d9-4ed9-ad12-ef03283d3576 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:55.612560 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a3056-2667-4899-bab1-455debc2657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:55.613196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a3056-2667-4899-bab1-455debc2657c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:55.613474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a3056-2667-4899-bab1-455debc2657c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:55.613742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a3056-2667-4899-bab1-455debc2657c None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:55.614360 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a3056-2667-4899-bab1-455debc2657c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:55.614677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a3056-2667-4899-bab1-455debc2657c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:55.616087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001a3056-2667-4899-bab1-455debc2657c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:55.625732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 490/986] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 22 21:57:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:55.626868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860b712-567c-462d-87ae-9bdbb71d35c4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:55.728760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8860b712-567c-462d-87ae-9bdbb71d35c4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:55.764175 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d4d1c-49d5-4946-ab79-42f473fcb989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:55.764867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d4d1c-49d5-4946-ab79-42f473fcb989 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:55.765238 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d4d1c-49d5-4946-ab79-42f473fcb989 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:55.765577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d4d1c-49d5-4946-ab79-42f473fcb989 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:55.766245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d4d1c-49d5-4946-ab79-42f473fcb989 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:55.766607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d4d1c-49d5-4946-ab79-42f473fcb989 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:55.768015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d4d1c-49d5-4946-ab79-42f473fcb989 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:55.773864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 497/987] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 22 21:57:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:55.779519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a3056-2667-4899-bab1-455debc2657c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:55.848851 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:55.880614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a3056-2667-4899-bab1-455debc2657c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:55.913128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affea377-96c4-4494-b751-cc943d2674a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:55.913786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affea377-96c4-4494-b751-cc943d2674a4 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:55.914126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affea377-96c4-4494-b751-cc943d2674a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:55.914460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affea377-96c4-4494-b751-cc943d2674a4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:55.915214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affea377-96c4-4494-b751-cc943d2674a4 None admin] RBAC: Adding query filter params (interface=admin, service_id=bb979e6ccb8a408d83ec3fe282c31842, region_id=RegionOne) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:55.915587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affea377-96c4-4494-b751-cc943d2674a4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:55.917024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affea377-96c4-4494-b751-cc943d2674a4 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:55.924528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 491/988] 10.222.0.25 () {56 vars in 1249 bytes} [Thu Sep 22 21:57:55 2022] GET /identity/v3/endpoints?service_id=bb979e6ccb8a408d83ec3fe282c31842&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:55.930541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d4d1c-49d5-4946-ab79-42f473fcb989 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:55.989047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:56.030344 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d4d1c-49d5-4946-ab79-42f473fcb989 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:56.075752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6056f8-ef0c-4f7e-b4be-a836014a773d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:56.076387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6056f8-ef0c-4f7e-b4be-a836014a773d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:56.076660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6056f8-ef0c-4f7e-b4be-a836014a773d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:56.076932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6056f8-ef0c-4f7e-b4be-a836014a773d None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:56.077559 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6056f8-ef0c-4f7e-b4be-a836014a773d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:56.078911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6056f8-ef0c-4f7e-b4be-a836014a773d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:56.093205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 498/989] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 22 21:57:55 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 21:57:56.223816 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ef952-b7aa-4013-9196-7c2d166f4c1d None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:56.224018 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ef952-b7aa-4013-9196-7c2d166f4c1d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:56.224272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ef952-b7aa-4013-9196-7c2d166f4c1d None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:56.226307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 492/990] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:56.232250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad4df4-2716-4551-a016-e695e1c90b5e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:56.232537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad4df4-2716-4551-a016-e695e1c90b5e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:56.232808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad4df4-2716-4551-a016-e695e1c90b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:56.252215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-faad4df4-2716-4551-a016-e695e1c90b5e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:56.253463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-faad4df4-2716-4551-a016-e695e1c90b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:56.324041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 499/991] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:56.595328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ef952-b7aa-4013-9196-7c2d166f4c1d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:56.699907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767ef952-b7aa-4013-9196-7c2d166f4c1d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:56.734354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1de41d-a14f-488f-a86f-7b7e850bbb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:56.734978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1de41d-a14f-488f-a86f-7b7e850bbb46 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:56.735252 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1de41d-a14f-488f-a86f-7b7e850bbb46 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:56.735519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1de41d-a14f-488f-a86f-7b7e850bbb46 None admin] PATH_INFO: `/v3/services/image` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:56.736093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1de41d-a14f-488f-a86f-7b7e850bbb46 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:56.737462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1de41d-a14f-488f-a86f-7b7e850bbb46 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:56.739177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-ba1de41d-a14f-488f-a86f-7b7e850bbb46 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:56.748334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:56.749631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 21:57:56.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:56.751762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 493/992] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 22 21:57:56 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:56.755237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad4df4-2716-4551-a016-e695e1c90b5e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:56.810457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:56.855337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad4df4-2716-4551-a016-e695e1c90b5e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:56.889779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead33a1-52f9-45ab-884c-0cfaea941590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:56.890377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead33a1-52f9-45ab-884c-0cfaea941590 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:56.890648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead33a1-52f9-45ab-884c-0cfaea941590 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:56.890911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead33a1-52f9-45ab-884c-0cfaea941590 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:56.891527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead33a1-52f9-45ab-884c-0cfaea941590 None admin] RBAC: Adding query filter params (name=image) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:56.891825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead33a1-52f9-45ab-884c-0cfaea941590 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:56.893204 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ead33a1-52f9-45ab-884c-0cfaea941590 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:56.902842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 500/993] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:56.904113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1de41d-a14f-488f-a86f-7b7e850bbb46 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:57.012397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1de41d-a14f-488f-a86f-7b7e850bbb46 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:57.043090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b40f9-b292-4dd9-8310-a87f2b37bf5e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:57.043427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b40f9-b292-4dd9-8310-a87f2b37bf5e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:57.043654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b40f9-b292-4dd9-8310-a87f2b37bf5e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:57.045140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 501/994] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:57:57.045967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7adde0-da42-4fa0-8fa7-880719839557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:57.046562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7adde0-da42-4fa0-8fa7-880719839557 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:57.046837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7adde0-da42-4fa0-8fa7-880719839557 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:57.047104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7adde0-da42-4fa0-8fa7-880719839557 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:57.047717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7adde0-da42-4fa0-8fa7-880719839557 None admin] RBAC: Adding query filter params (type=image) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:57.048012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7adde0-da42-4fa0-8fa7-880719839557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:57.049410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7adde0-da42-4fa0-8fa7-880719839557 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:57.049747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e4390-6855-4769-8638-b1217bd58841 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:57.050031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e4390-6855-4769-8638-b1217bd58841 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:57.050305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e4390-6855-4769-8638-b1217bd58841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:57.058940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 494/995] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:57:56 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:57.072409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f01e4390-6855-4769-8638-b1217bd58841 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:57.073668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f01e4390-6855-4769-8638-b1217bd58841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:57.149209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 502/996] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:57:57.417176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7adde0-da42-4fa0-8fa7-880719839557 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:57.479405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:57.518903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7adde0-da42-4fa0-8fa7-880719839557 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:57.554584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d05bd6-100e-422f-8d14-3cdd5381b863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:57.555565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05bd6-100e-422f-8d14-3cdd5381b863 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:57.556036 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05bd6-100e-422f-8d14-3cdd5381b863 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:57.556531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d05bd6-100e-422f-8d14-3cdd5381b863 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:57.557362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d05bd6-100e-422f-8d14-3cdd5381b863 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:57.558903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d05bd6-100e-422f-8d14-3cdd5381b863 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a7d05bd6-100e-422f-8d14-3cdd5381b863 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:57.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:57.568556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 21:57:57.569471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:57:57.570885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 495/997] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:57:57 2022] GET /identity/v3/services/compute => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:57.576392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e4390-6855-4769-8638-b1217bd58841 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:57.626296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:57.678663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e4390-6855-4769-8638-b1217bd58841 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:57.714348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9354ba-f759-4eee-83fb-7082c2525c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:57.714952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9354ba-f759-4eee-83fb-7082c2525c11 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:57.715229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9354ba-f759-4eee-83fb-7082c2525c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:57.715494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9354ba-f759-4eee-83fb-7082c2525c11 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:57.716114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9354ba-f759-4eee-83fb-7082c2525c11 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:57.716411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9354ba-f759-4eee-83fb-7082c2525c11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:57.717835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9354ba-f759-4eee-83fb-7082c2525c11 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:57.727280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 503/998] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:57:57 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:57.728432 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d05bd6-100e-422f-8d14-3cdd5381b863 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:57.830359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d05bd6-100e-422f-8d14-3cdd5381b863 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:57.856406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:57.866484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf78d91-7408-4428-a299-cade41a23da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:57.867091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf78d91-7408-4428-a299-cade41a23da8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:57.867367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf78d91-7408-4428-a299-cade41a23da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:57.867637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf78d91-7408-4428-a299-cade41a23da8 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:57.868259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf78d91-7408-4428-a299-cade41a23da8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:57.868559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf78d91-7408-4428-a299-cade41a23da8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:57.869951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf78d91-7408-4428-a299-cade41a23da8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:57.874896 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 496/999] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:57:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:57:57.880499 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9354ba-f759-4eee-83fb-7082c2525c11 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:57.984464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9354ba-f759-4eee-83fb-7082c2525c11 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:57.998440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:58.020762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f080158-339f-4117-9c64-3c8efb63480c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:58.021407 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f080158-339f-4117-9c64-3c8efb63480c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:58.021774 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f080158-339f-4117-9c64-3c8efb63480c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:58.022052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f080158-339f-4117-9c64-3c8efb63480c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:58.022933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f080158-339f-4117-9c64-3c8efb63480c None admin] RBAC: Adding query filter params (interface=public, service_id=90bcd0145d8c4b029a57468376b8f532, region_id=RegionOne) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:58.023250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f080158-339f-4117-9c64-3c8efb63480c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:58.024614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f080158-339f-4117-9c64-3c8efb63480c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:58.034283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 504/1000] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 22 21:57:57 2022] GET /identity/v3/endpoints?service_id=90bcd0145d8c4b029a57468376b8f532&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:58.035418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf78d91-7408-4428-a299-cade41a23da8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:58.145479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf78d91-7408-4428-a299-cade41a23da8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:58.182724 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41fa682-c474-4089-b891-563993d02e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:58.183341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fa682-c474-4089-b891-563993d02e6a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:58.183661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fa682-c474-4089-b891-563993d02e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:58.183940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41fa682-c474-4089-b891-563993d02e6a None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:58.184564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41fa682-c474-4089-b891-563993d02e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:58.186002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41fa682-c474-4089-b891-563993d02e6a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:58.199578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 497/1001] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 22 21:57:58 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 21:57:58.287630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d94b0a-3714-4f19-91ff-55ab4bba90b6 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:58.288175 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d94b0a-3714-4f19-91ff-55ab4bba90b6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:58.288364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d94b0a-3714-4f19-91ff-55ab4bba90b6 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:58.290320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 505/1002] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:57:58.295986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e648c7-6db3-4348-acc4-3bec7c133305 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:58.296276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e648c7-6db3-4348-acc4-3bec7c133305 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:58.296551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e648c7-6db3-4348-acc4-3bec7c133305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:58.315945 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a4e648c7-6db3-4348-acc4-3bec7c133305 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:58.317202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a4e648c7-6db3-4348-acc4-3bec7c133305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:58.372257 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:58.392479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 498/1003] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:57:58.693016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d94b0a-3714-4f19-91ff-55ab4bba90b6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:58.810262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d94b0a-3714-4f19-91ff-55ab4bba90b6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:58.855949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820e94f-16a3-44b8-b307-311e539f5405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:58.856891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820e94f-16a3-44b8-b307-311e539f5405 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:58.857341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820e94f-16a3-44b8-b307-311e539f5405 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:58.857752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820e94f-16a3-44b8-b307-311e539f5405 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:58.858468 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820e94f-16a3-44b8-b307-311e539f5405 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:58.859981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820e94f-16a3-44b8-b307-311e539f5405 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1820e94f-16a3-44b8-b307-311e539f5405 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:57:58.868921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:57:58.870154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 22 21:57:58.871172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:57:58.873627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 506/1004] 10.222.0.25 () {56 vars in 1126 bytes} [Thu Sep 22 21:57:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:57:58.879253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e648c7-6db3-4348-acc4-3bec7c133305 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:58.982029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e648c7-6db3-4348-acc4-3bec7c133305 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:59.014866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c635bb-e6c3-4129-8d11-1b078dd55617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:59.015459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c635bb-e6c3-4129-8d11-1b078dd55617 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:59.015731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c635bb-e6c3-4129-8d11-1b078dd55617 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:59.015999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c635bb-e6c3-4129-8d11-1b078dd55617 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:59.016614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c635bb-e6c3-4129-8d11-1b078dd55617 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:59.016916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c635bb-e6c3-4129-8d11-1b078dd55617 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:59.018324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c635bb-e6c3-4129-8d11-1b078dd55617 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:59.023656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 499/1005] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 22 21:57:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:59.029067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820e94f-16a3-44b8-b307-311e539f5405 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:59.080562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:59.152130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820e94f-16a3-44b8-b307-311e539f5405 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:59.191480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2169cf-a0e3-4014-b37b-73a4f6e9e623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:59.192078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2169cf-a0e3-4014-b37b-73a4f6e9e623 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:59.192349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2169cf-a0e3-4014-b37b-73a4f6e9e623 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:59.192615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2169cf-a0e3-4014-b37b-73a4f6e9e623 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:59.193545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2169cf-a0e3-4014-b37b-73a4f6e9e623 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:57:59.193847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2169cf-a0e3-4014-b37b-73a4f6e9e623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:59.195766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2169cf-a0e3-4014-b37b-73a4f6e9e623 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:59.200702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 507/1006] 10.222.0.25 () {56 vars in 1122 bytes} [Thu Sep 22 21:57:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:57:59.206223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c635bb-e6c3-4129-8d11-1b078dd55617 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:59.252721 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d38cb-9c6f-405f-9d7f-c6bcf87df54a None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:59.253041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d38cb-9c6f-405f-9d7f-c6bcf87df54a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:59.253307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91d38cb-9c6f-405f-9d7f-c6bcf87df54a None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:59.254771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 508/1007] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:57:59.256756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:59.259692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf5a0c-7d72-499a-8917-902b91bcc208 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:59.259971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf5a0c-7d72-499a-8917-902b91bcc208 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:59.260247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf5a0c-7d72-499a-8917-902b91bcc208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:59.278279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-66bf5a0c-7d72-499a-8917-902b91bcc208 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:57:59.279545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-66bf5a0c-7d72-499a-8917-902b91bcc208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:57:59.310503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c635bb-e6c3-4129-8d11-1b078dd55617 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:59.345722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e153079-535e-491c-85ce-bd5256f31425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:59.346338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e153079-535e-491c-85ce-bd5256f31425 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:59.346623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e153079-535e-491c-85ce-bd5256f31425 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:59.346891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e153079-535e-491c-85ce-bd5256f31425 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:59.347561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e153079-535e-491c-85ce-bd5256f31425 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:59.348918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e153079-535e-491c-85ce-bd5256f31425 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:59.353661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 509/1008] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:57:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:57:59.373143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 500/1009] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 21:57:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:57:59.628659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf5a0c-7d72-499a-8917-902b91bcc208 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:57:59.662164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:57:59.731369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf5a0c-7d72-499a-8917-902b91bcc208 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:57:59.776393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583eda3b-695a-4c26-9f3f-95ffcd49708e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:57:59.777005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eda3b-695a-4c26-9f3f-95ffcd49708e None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:57:59.777307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eda3b-695a-4c26-9f3f-95ffcd49708e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:57:59.777573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583eda3b-695a-4c26-9f3f-95ffcd49708e None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:57:59.778236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583eda3b-695a-4c26-9f3f-95ffcd49708e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:57:59.779593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583eda3b-695a-4c26-9f3f-95ffcd49708e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:57:59.801167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 510/1010] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:57:59 2022] POST /identity/v3/services => generated 242 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:58:00.410390 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105959f-796a-4768-a2b2-8f5987b847e3 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:00.410890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105959f-796a-4768-a2b2-8f5987b847e3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:00.410890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105959f-796a-4768-a2b2-8f5987b847e3 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:00.412330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 501/1011] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:00.418024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bccfb1-84d2-4fb6-8e91-1221cdb7f8dd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:00.418404 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bccfb1-84d2-4fb6-8e91-1221cdb7f8dd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:00.418568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bccfb1-84d2-4fb6-8e91-1221cdb7f8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:00.436518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f8bccfb1-84d2-4fb6-8e91-1221cdb7f8dd None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:00.437726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f8bccfb1-84d2-4fb6-8e91-1221cdb7f8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:00.510833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 511/1012] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:00 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:00.788165 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105959f-796a-4768-a2b2-8f5987b847e3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:00.888968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105959f-796a-4768-a2b2-8f5987b847e3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:00.898086 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:00.924998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb558a-e1b0-473b-9493-f5206b5475b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:00.925638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb558a-e1b0-473b-9493-f5206b5475b3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:00.925916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb558a-e1b0-473b-9493-f5206b5475b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:00.926185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb558a-e1b0-473b-9493-f5206b5475b3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:00.926756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb558a-e1b0-473b-9493-f5206b5475b3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:00.928100 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71eb558a-e1b0-473b-9493-f5206b5475b3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-71eb558a-e1b0-473b-9493-f5206b5475b3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:00.937522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:00.938504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 22 21:58:00.939376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:00.941189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 502/1013] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:58:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:00.945173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bccfb1-84d2-4fb6-8e91-1221cdb7f8dd None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:00.962750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:01.047350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bccfb1-84d2-4fb6-8e91-1221cdb7f8dd None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:01.086859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a33200-fdcd-4b8d-aa4d-53edda6dd50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:01.087472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33200-fdcd-4b8d-aa4d-53edda6dd50b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:01.087749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33200-fdcd-4b8d-aa4d-53edda6dd50b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:01.088017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a33200-fdcd-4b8d-aa4d-53edda6dd50b None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:01.088896 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a33200-fdcd-4b8d-aa4d-53edda6dd50b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:01.089226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a33200-fdcd-4b8d-aa4d-53edda6dd50b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:01.090579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a33200-fdcd-4b8d-aa4d-53edda6dd50b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:01.095650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 512/1014] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:58:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:01.101422 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb558a-e1b0-473b-9493-f5206b5475b3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:01.206073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb558a-e1b0-473b-9493-f5206b5475b3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:01.239669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e850d-80a1-4a66-8860-52072b434106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:01.240272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e850d-80a1-4a66-8860-52072b434106 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:01.240550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e850d-80a1-4a66-8860-52072b434106 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:01.241361 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e850d-80a1-4a66-8860-52072b434106 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:01.241989 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e850d-80a1-4a66-8860-52072b434106 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:01.242287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e850d-80a1-4a66-8860-52072b434106 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:01.243648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e850d-80a1-4a66-8860-52072b434106 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:01.253262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 503/1015] 10.222.0.25 () {56 vars in 1112 bytes} [Thu Sep 22 21:58:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:01.254433 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a33200-fdcd-4b8d-aa4d-53edda6dd50b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:01.335879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:01.356551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a33200-fdcd-4b8d-aa4d-53edda6dd50b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:01.390763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9dd6b2-c817-418e-afda-9e716c818a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:01.391362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dd6b2-c817-418e-afda-9e716c818a3f None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:01.391634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dd6b2-c817-418e-afda-9e716c818a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:01.391897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9dd6b2-c817-418e-afda-9e716c818a3f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:01.392551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dd6b2-c817-418e-afda-9e716c818a3f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:01.393933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9dd6b2-c817-418e-afda-9e716c818a3f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:01.417147 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 513/1016] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 21:58:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:58:01.570922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4772a-1c55-4bd3-aa8a-0b801bcdb262 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:01.571477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4772a-1c55-4bd3-aa8a-0b801bcdb262 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:01.571477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4772a-1c55-4bd3-aa8a-0b801bcdb262 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:01.572992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 504/1017] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:01.578550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5d341-25cc-443d-b8d9-537551e4d5b1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:01.579005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5d341-25cc-443d-b8d9-537551e4d5b1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:01.579097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5d341-25cc-443d-b8d9-537551e4d5b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:01.597800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-43f5d341-25cc-443d-b8d9-537551e4d5b1 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:01.598880 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-43f5d341-25cc-443d-b8d9-537551e4d5b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:01.676523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 514/1018] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:01 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:01.949532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4772a-1c55-4bd3-aa8a-0b801bcdb262 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:01.985509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:02.026666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e048af-26c8-4d9b-84ac-d688054bf972 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:02.026951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e048af-26c8-4d9b-84ac-d688054bf972 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:02.027222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e048af-26c8-4d9b-84ac-d688054bf972 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:02.028730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 515/1019] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:58:02.033403 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365644ec-38da-486b-ae58-58eae9922fe8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:02.033686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365644ec-38da-486b-ae58-58eae9922fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:02.033958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365644ec-38da-486b-ae58-58eae9922fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:02.052182 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-365644ec-38da-486b-ae58-58eae9922fe8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:02.053908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-365644ec-38da-486b-ae58-58eae9922fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:02.071166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4772a-1c55-4bd3-aa8a-0b801bcdb262 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:02.108500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba4a7a-2870-4680-b003-12dcdc7a1e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:02.109131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba4a7a-2870-4680-b003-12dcdc7a1e8e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:02.109656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba4a7a-2870-4680-b003-12dcdc7a1e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:02.109656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba4a7a-2870-4680-b003-12dcdc7a1e8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:02.110763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba4a7a-2870-4680-b003-12dcdc7a1e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:02.112180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ba4a7a-2870-4680-b003-12dcdc7a1e8e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:02.117146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 505/1020] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:02.119805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba4a7a-2870-4680-b003-12dcdc7a1e8e None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:02.122145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:02.135768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 516/1021] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:02 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:58:02.238825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ba4a7a-2870-4680-b003-12dcdc7a1e8e None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:02.241740 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:02.273665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4252a-e4c0-45d2-899d-c2d6e191e1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:02.274261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4252a-e4c0-45d2-899d-c2d6e191e1fc None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:02.274534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4252a-e4c0-45d2-899d-c2d6e191e1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:02.274801 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d4252a-e4c0-45d2-899d-c2d6e191e1fc None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:02.275436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4252a-e4c0-45d2-899d-c2d6e191e1fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:02.276881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d4252a-e4c0-45d2-899d-c2d6e191e1fc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:02.313131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 506/1022] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:58:02 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:58:02.411646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365644ec-38da-486b-ae58-58eae9922fe8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:02.522236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365644ec-38da-486b-ae58-58eae9922fe8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:02.557824 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7635a8b-8e10-446a-91ab-f932aaca72cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:02.558451 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7635a8b-8e10-446a-91ab-f932aaca72cf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:02.558723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7635a8b-8e10-446a-91ab-f932aaca72cf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:02.558989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7635a8b-8e10-446a-91ab-f932aaca72cf None admin] PATH_INFO: `/v3/services/image` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:02.559565 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7635a8b-8e10-446a-91ab-f932aaca72cf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:02.560901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7635a8b-8e10-446a-91ab-f932aaca72cf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-d7635a8b-8e10-446a-91ab-f932aaca72cf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:02.569493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:02.570474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:02.571898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 517/1023] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 22 21:58:02 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:02.576508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4252a-e4c0-45d2-899d-c2d6e191e1fc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:02.678341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d4252a-e4c0-45d2-899d-c2d6e191e1fc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:02.712939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63340d3f-5b3d-4ff1-bfdf-a674f8a8a2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:02.713815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63340d3f-5b3d-4ff1-bfdf-a674f8a8a2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:02.714085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63340d3f-5b3d-4ff1-bfdf-a674f8a8a2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:02.714350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63340d3f-5b3d-4ff1-bfdf-a674f8a8a2f0 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:02.714968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63340d3f-5b3d-4ff1-bfdf-a674f8a8a2f0 None admin] RBAC: Adding query filter params (name=image) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:02.715266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63340d3f-5b3d-4ff1-bfdf-a674f8a8a2f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:02.716611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63340d3f-5b3d-4ff1-bfdf-a674f8a8a2f0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:02.726846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 507/1024] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:02.728050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7635a8b-8e10-446a-91ab-f932aaca72cf None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:02.785469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:02.832107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7635a8b-8e10-446a-91ab-f932aaca72cf None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:02.866298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62aee3-a64b-4146-b1a3-f671ad776dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:02.866903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62aee3-a64b-4146-b1a3-f671ad776dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:02.867172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62aee3-a64b-4146-b1a3-f671ad776dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:02.867440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc62aee3-a64b-4146-b1a3-f671ad776dd9 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:02.868057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62aee3-a64b-4146-b1a3-f671ad776dd9 None admin] RBAC: Adding query filter params (type=image) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:02.868353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62aee3-a64b-4146-b1a3-f671ad776dd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:02.869740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc62aee3-a64b-4146-b1a3-f671ad776dd9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:02.874749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 518/1025] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:02 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:02.880429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63340d3f-5b3d-4ff1-bfdf-a674f8a8a2f0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:02.903721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:02.985404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63340d3f-5b3d-4ff1-bfdf-a674f8a8a2f0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:03.019203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39601e-bd38-4031-8c0a-635ba439b4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:03.019805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39601e-bd38-4031-8c0a-635ba439b4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:03.020084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39601e-bd38-4031-8c0a-635ba439b4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:03.020358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39601e-bd38-4031-8c0a-635ba439b4f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:03.021059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39601e-bd38-4031-8c0a-635ba439b4f8 None admin] RBAC: Adding query filter params (interface=public, service_id=a26e19c8aba740b68178a2b198df9065, region_id=RegionOne) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:03.021398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39601e-bd38-4031-8c0a-635ba439b4f8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:03.023009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39601e-bd38-4031-8c0a-635ba439b4f8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:03.032635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 508/1026] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 22 21:58:02 2022] GET /identity/v3/endpoints?service_id=a26e19c8aba740b68178a2b198df9065&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:03.033716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62aee3-a64b-4146-b1a3-f671ad776dd9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:03.145469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc62aee3-a64b-4146-b1a3-f671ad776dd9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:03.165564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:03.180937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277fc42-b7b5-4a7b-8e37-34cda30b4d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:03.181573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277fc42-b7b5-4a7b-8e37-34cda30b4d97 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:03.181854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277fc42-b7b5-4a7b-8e37-34cda30b4d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:03.182124 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277fc42-b7b5-4a7b-8e37-34cda30b4d97 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:03.182729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277fc42-b7b5-4a7b-8e37-34cda30b4d97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:03.184082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277fc42-b7b5-4a7b-8e37-34cda30b4d97 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:03.197433 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 519/1027] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 22 21:58:03 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 21:58:03.712707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39807f6-3e57-43cf-b0ba-4feb53cea424 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:03.712707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39807f6-3e57-43cf-b0ba-4feb53cea424 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:03.712707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39807f6-3e57-43cf-b0ba-4feb53cea424 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:03.714113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 509/1028] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:03.717970 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257192f3-42ac-4fa3-8105-3d6dcb8e2f93 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:03.718253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257192f3-42ac-4fa3-8105-3d6dcb8e2f93 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:03.718519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257192f3-42ac-4fa3-8105-3d6dcb8e2f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:03.736993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-257192f3-42ac-4fa3-8105-3d6dcb8e2f93 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:03.738198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-257192f3-42ac-4fa3-8105-3d6dcb8e2f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:03.804120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 520/1029] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:03 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:04.070416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39807f6-3e57-43cf-b0ba-4feb53cea424 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:04.173339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:04.193858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39807f6-3e57-43cf-b0ba-4feb53cea424 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:04.232915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62533af7-778d-418c-8e38-1d4f456208f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:04.233626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62533af7-778d-418c-8e38-1d4f456208f5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:04.233912 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62533af7-778d-418c-8e38-1d4f456208f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:04.234178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62533af7-778d-418c-8e38-1d4f456208f5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:04.246908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62533af7-778d-418c-8e38-1d4f456208f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-62533af7-778d-418c-8e38-1d4f456208f5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:58:04.252488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:04.253473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:58:04.254305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:04.259215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 510/1030] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 22 21:58:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:04.260396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257192f3-42ac-4fa3-8105-3d6dcb8e2f93 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:04.361429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257192f3-42ac-4fa3-8105-3d6dcb8e2f93 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:04.376075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:04.397773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388bfba-ba72-48c5-83ad-116949b73e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:04.398376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388bfba-ba72-48c5-83ad-116949b73e2e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:04.398642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388bfba-ba72-48c5-83ad-116949b73e2e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:04.398909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388bfba-ba72-48c5-83ad-116949b73e2e None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:04.399564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388bfba-ba72-48c5-83ad-116949b73e2e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:04.399860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388bfba-ba72-48c5-83ad-116949b73e2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:04.401241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388bfba-ba72-48c5-83ad-116949b73e2e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:04.414903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 521/1031] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 22 21:58:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:04.417160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62533af7-778d-418c-8e38-1d4f456208f5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:04.525651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62533af7-778d-418c-8e38-1d4f456208f5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:04.533823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095d414-aea7-4645-b700-f6f90fd87ee5 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:04.533823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095d414-aea7-4645-b700-f6f90fd87ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:04.534164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095d414-aea7-4645-b700-f6f90fd87ee5 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:04.535452 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 522/1032] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:58:04.541508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fd3ba-6bb6-485f-ade4-bdae19e0239b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:04.541790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fd3ba-6bb6-485f-ade4-bdae19e0239b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:04.542066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298fd3ba-6bb6-485f-ade4-bdae19e0239b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:04.560514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-298fd3ba-6bb6-485f-ade4-bdae19e0239b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:04.561711 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-298fd3ba-6bb6-485f-ade4-bdae19e0239b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:04.567590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9a6c3-ef31-412f-b103-bc5deaa34147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:04.568457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9a6c3-ef31-412f-b103-bc5deaa34147 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:04.569489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9a6c3-ef31-412f-b103-bc5deaa34147 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:04.569985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce9a6c3-ef31-412f-b103-bc5deaa34147 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:04.583273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9a6c3-ef31-412f-b103-bc5deaa34147 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:04.585718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce9a6c3-ef31-412f-b103-bc5deaa34147 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:04.589021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-dce9a6c3-ef31-412f-b103-bc5deaa34147 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:58:04.603731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:04.604699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:58:04.610274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:58:04.610274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:04.610274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:04.610274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 511/1033] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:04.610521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9a6c3-ef31-412f-b103-bc5deaa34147 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:04.637223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 523/1034] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:58:04.711264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce9a6c3-ef31-412f-b103-bc5deaa34147 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:04.745151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018f281-4f5d-4f47-991c-5cbcaba71355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:04.745734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018f281-4f5d-4f47-991c-5cbcaba71355 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:04.746013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018f281-4f5d-4f47-991c-5cbcaba71355 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:04.746284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018f281-4f5d-4f47-991c-5cbcaba71355 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:04.746915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018f281-4f5d-4f47-991c-5cbcaba71355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:04.747216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018f281-4f5d-4f47-991c-5cbcaba71355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:04.748578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018f281-4f5d-4f47-991c-5cbcaba71355 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:04.764485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 512/1035] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:58:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:04.766106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fd3ba-6bb6-485f-ade4-bdae19e0239b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:04.868763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298fd3ba-6bb6-485f-ade4-bdae19e0239b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:04.877172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:04.906574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e06bf3-936b-44b7-8bac-4d15f2430bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:04.907120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e06bf3-936b-44b7-8bac-4d15f2430bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:04.907397 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e06bf3-936b-44b7-8bac-4d15f2430bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:04.907666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e06bf3-936b-44b7-8bac-4d15f2430bd7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:04.913163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018f281-4f5d-4f47-991c-5cbcaba71355 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:04.915121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e06bf3-936b-44b7-8bac-4d15f2430bd7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:04.917655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e06bf3-936b-44b7-8bac-4d15f2430bd7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-04e06bf3-936b-44b7-8bac-4d15f2430bd7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:58:04.931814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:04.932987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:04.934134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:04.934134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:04.934134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:58:04.934134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:58:04.934134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:58:04.934134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:58:04.934134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:58:04.934134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:58:04.934134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:58:04.934134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:04.934134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 524/1036] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 22 21:58:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:04.938447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e06bf3-936b-44b7-8bac-4d15f2430bd7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:04.978399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:05.030751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018f281-4f5d-4f47-991c-5cbcaba71355 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:05.048178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e06bf3-936b-44b7-8bac-4d15f2430bd7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:05.075729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98c56c-cd2d-45a0-975a-0282e8407ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:05.076362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98c56c-cd2d-45a0-975a-0282e8407ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:05.076646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98c56c-cd2d-45a0-975a-0282e8407ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:05.076918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f98c56c-cd2d-45a0-975a-0282e8407ca8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:05.078062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98c56c-cd2d-45a0-975a-0282e8407ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:05.079499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f98c56c-cd2d-45a0-975a-0282e8407ca8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:05.081242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 513/1037] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:05.086637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98c56c-cd2d-45a0-975a-0282e8407ca8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:05.090104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e263662-97e0-4452-89cb-a682db0dc273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:05.090711 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e263662-97e0-4452-89cb-a682db0dc273 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:05.091011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e263662-97e0-4452-89cb-a682db0dc273 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:05.091276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e263662-97e0-4452-89cb-a682db0dc273 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:05.091992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e263662-97e0-4452-89cb-a682db0dc273 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:05.092294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e263662-97e0-4452-89cb-a682db0dc273 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:05.093818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e263662-97e0-4452-89cb-a682db0dc273 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:05.117141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 525/1038] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 22 21:58:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:05.118784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e263662-97e0-4452-89cb-a682db0dc273 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:05.146154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:05.195865 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f98c56c-cd2d-45a0-975a-0282e8407ca8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:05.222532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:05.227557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e263662-97e0-4452-89cb-a682db0dc273 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:05.234814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa01a3b7-c49e-4efa-bda7-09d3595a7b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:05.235409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01a3b7-c49e-4efa-bda7-09d3595a7b93 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:05.235691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01a3b7-c49e-4efa-bda7-09d3595a7b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:05.235959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa01a3b7-c49e-4efa-bda7-09d3595a7b93 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:05.236604 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01a3b7-c49e-4efa-bda7-09d3595a7b93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:05.238050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa01a3b7-c49e-4efa-bda7-09d3595a7b93 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:05.261944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 514/1039] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:58:05 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:58:05.276539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61ebe7-7b24-4513-9fa0-ebd8f2e7d624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:05.277218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61ebe7-7b24-4513-9fa0-ebd8f2e7d624 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:05.277499 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61ebe7-7b24-4513-9fa0-ebd8f2e7d624 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:05.277770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61ebe7-7b24-4513-9fa0-ebd8f2e7d624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:05.357252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61ebe7-7b24-4513-9fa0-ebd8f2e7d624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:05.358722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61ebe7-7b24-4513-9fa0-ebd8f2e7d624 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:05.385562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:05.410281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0957e-2497-4bfc-a134-7da25afcb56a None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:05.410598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0957e-2497-4bfc-a134-7da25afcb56a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:05.410834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0957e-2497-4bfc-a134-7da25afcb56a None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:05.413302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 515/1040] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:58:05.418854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045a576-c649-4568-8e3d-211a9f96ed68 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:05.419261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045a576-c649-4568-8e3d-211a9f96ed68 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:05.419606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045a576-c649-4568-8e3d-211a9f96ed68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:05.438794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1045a576-c649-4568-8e3d-211a9f96ed68 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:05.439999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1045a576-c649-4568-8e3d-211a9f96ed68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:05.480790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 526/1041] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:58:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:58:05.481680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61ebe7-7b24-4513-9fa0-ebd8f2e7d624 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:05.523856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 516/1042] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:05 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:05.592187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61ebe7-7b24-4513-9fa0-ebd8f2e7d624 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:05.616752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:05.628387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335abd74-0796-441f-86f7-e7beb64a3b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:05.629010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335abd74-0796-441f-86f7-e7beb64a3b3b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:05.629313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335abd74-0796-441f-86f7-e7beb64a3b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:05.629582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335abd74-0796-441f-86f7-e7beb64a3b3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:05.639950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335abd74-0796-441f-86f7-e7beb64a3b3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:05.641422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335abd74-0796-441f-86f7-e7beb64a3b3b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-335abd74-0796-441f-86f7-e7beb64a3b3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:58:05.657419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:05.658381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:58:05.659293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:58:05.659293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:05.659293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:05.659293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 527/1043] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:05.666260 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045a576-c649-4568-8e3d-211a9f96ed68 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:05.767215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045a576-c649-4568-8e3d-211a9f96ed68 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:05.778877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:05.800885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335abd74-0796-441f-86f7-e7beb64a3b3b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:05.809312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67304852-b353-4559-9804-4bf3cd405169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:05.809925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67304852-b353-4559-9804-4bf3cd405169 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:05.810207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67304852-b353-4559-9804-4bf3cd405169 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:05.810478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67304852-b353-4559-9804-4bf3cd405169 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:05.811544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67304852-b353-4559-9804-4bf3cd405169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:05.811851 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67304852-b353-4559-9804-4bf3cd405169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:05.813626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67304852-b353-4559-9804-4bf3cd405169 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:05.830759 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 517/1044] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:58:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:05.832435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67304852-b353-4559-9804-4bf3cd405169 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:05.914901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335abd74-0796-441f-86f7-e7beb64a3b3b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:05.944242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67304852-b353-4559-9804-4bf3cd405169 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:05.951908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aab9da-4ecb-47a9-8a75-dc624dcb982d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:05.952546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab9da-4ecb-47a9-8a75-dc624dcb982d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:05.952821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab9da-4ecb-47a9-8a75-dc624dcb982d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:05.953092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9aab9da-4ecb-47a9-8a75-dc624dcb982d None admin] PATH_INFO: `/v3/services/image` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:05.953696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aab9da-4ecb-47a9-8a75-dc624dcb982d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:05.955050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9aab9da-4ecb-47a9-8a75-dc624dcb982d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b9aab9da-4ecb-47a9-8a75-dc624dcb982d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:05.964312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:05.965350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 22 21:58:05.966247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:05.970390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 528/1045] 10.222.0.25 () {56 vars in 1105 bytes} [Thu Sep 22 21:58:05 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:58:05.971222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aab9da-4ecb-47a9-8a75-dc624dcb982d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:05.983230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2ab4d-df53-45ba-b4fd-b0fd68aa140a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:05.984033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2ab4d-df53-45ba-b4fd-b0fd68aa140a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:05.984315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2ab4d-df53-45ba-b4fd-b0fd68aa140a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:05.984585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2ab4d-df53-45ba-b4fd-b0fd68aa140a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:05.992112 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2ab4d-df53-45ba-b4fd-b0fd68aa140a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:05.993635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2ab4d-df53-45ba-b4fd-b0fd68aa140a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-35a2ab4d-df53-45ba-b4fd-b0fd68aa140a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:58:06.004953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:06.005986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:06.006932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:58:06.006932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:58:06.006932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:58:06.006932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:58:06.006932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:58:06.006932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:58:06.006932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:58:06.006932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:06.011054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 518/1046] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:58:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:58:06.012297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2ab4d-df53-45ba-b4fd-b0fd68aa140a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:06.015681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:06.045690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:06.084975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9aab9da-4ecb-47a9-8a75-dc624dcb982d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:06.122651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0b4a0-57ef-4a47-8328-266f75cc98d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:06.122995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2ab4d-df53-45ba-b4fd-b0fd68aa140a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:06.123252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0b4a0-57ef-4a47-8328-266f75cc98d7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:06.123523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0b4a0-57ef-4a47-8328-266f75cc98d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:06.123788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0b4a0-57ef-4a47-8328-266f75cc98d7 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:06.124413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0b4a0-57ef-4a47-8328-266f75cc98d7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:06.124709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0b4a0-57ef-4a47-8328-266f75cc98d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:06.126098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0b4a0-57ef-4a47-8328-266f75cc98d7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:06.130983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 529/1047] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:06.136530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0b4a0-57ef-4a47-8328-266f75cc98d7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:06.161950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73b9b3-9f77-47d9-bb4f-e9dca92d060a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:06.163176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73b9b3-9f77-47d9-bb4f-e9dca92d060a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:06.163451 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73b9b3-9f77-47d9-bb4f-e9dca92d060a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:06.163723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf73b9b3-9f77-47d9-bb4f-e9dca92d060a None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:06.164392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73b9b3-9f77-47d9-bb4f-e9dca92d060a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:06.164691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73b9b3-9f77-47d9-bb4f-e9dca92d060a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:06.166127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf73b9b3-9f77-47d9-bb4f-e9dca92d060a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:06.179038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 519/1048] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:58:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:06.184844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73b9b3-9f77-47d9-bb4f-e9dca92d060a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:06.251873 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0b4a0-57ef-4a47-8328-266f75cc98d7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:06.254936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:06.280628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:06.289731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e632df-3e85-46e8-b0b6-eb5d5b4d4a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:06.290331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e632df-3e85-46e8-b0b6-eb5d5b4d4a2f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:06.290603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e632df-3e85-46e8-b0b6-eb5d5b4d4a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:06.290870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e632df-3e85-46e8-b0b6-eb5d5b4d4a2f None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:06.291485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e632df-3e85-46e8-b0b6-eb5d5b4d4a2f None admin] RBAC: Adding query filter params (type=image) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:06.291781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e632df-3e85-46e8-b0b6-eb5d5b4d4a2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:06.292372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf73b9b3-9f77-47d9-bb4f-e9dca92d060a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:06.293138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e632df-3e85-46e8-b0b6-eb5d5b4d4a2f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:06.303812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 530/1049] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:06.304558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e632df-3e85-46e8-b0b6-eb5d5b4d4a2f None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:06.332535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d438d3-01db-43b5-9ce4-2a9cb237710c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:06.333156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d438d3-01db-43b5-9ce4-2a9cb237710c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:06.333433 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d438d3-01db-43b5-9ce4-2a9cb237710c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:06.333707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d438d3-01db-43b5-9ce4-2a9cb237710c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:06.335155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d438d3-01db-43b5-9ce4-2a9cb237710c None admin] RBAC: Adding query filter params (role.id=f1c696334f084cccb674b2528a9fa65c, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=d011e54e12674eaeb82c68da628ecb9d) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:06.335471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d438d3-01db-43b5-9ce4-2a9cb237710c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:06.336890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d438d3-01db-43b5-9ce4-2a9cb237710c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:06.343403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 520/1050] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:58:06 2022] GET /identity/v3/role_assignments?user.id=d011e54e12674eaeb82c68da628ecb9d&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=f1c696334f084cccb674b2528a9fa65c => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:06.410750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e632df-3e85-46e8-b0b6-eb5d5b4d4a2f None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:06.447062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4c830-9f30-477c-95b1-2fcf7f31d071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:06.447682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4c830-9f30-477c-95b1-2fcf7f31d071 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:06.447953 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4c830-9f30-477c-95b1-2fcf7f31d071 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:06.448220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4c830-9f30-477c-95b1-2fcf7f31d071 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:06.448905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4c830-9f30-477c-95b1-2fcf7f31d071 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:06.450296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4c830-9f30-477c-95b1-2fcf7f31d071 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:06.474395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 531/1051] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 21:58:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 21:58:07.505149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f536299-5db8-4cec-8347-3ce6e4d77b51 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:07.505674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f536299-5db8-4cec-8347-3ce6e4d77b51 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:07.505674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f536299-5db8-4cec-8347-3ce6e4d77b51 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:07.507308 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 521/1052] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:07.512530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588ecfb-31ee-4439-b3b2-f7d39c2b998e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:07.512946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588ecfb-31ee-4439-b3b2-f7d39c2b998e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:07.513089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6588ecfb-31ee-4439-b3b2-f7d39c2b998e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:07.531607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6588ecfb-31ee-4439-b3b2-f7d39c2b998e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:07.532800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6588ecfb-31ee-4439-b3b2-f7d39c2b998e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:07.550985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:07.608394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 532/1053] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:07.879031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f536299-5db8-4cec-8347-3ce6e4d77b51 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:07.981024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f536299-5db8-4cec-8347-3ce6e4d77b51 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:08.015413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fcc91-318f-405f-8807-25e5370f53ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:08.016030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fcc91-318f-405f-8807-25e5370f53ed None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:08.016304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fcc91-318f-405f-8807-25e5370f53ed None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:08.016627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585fcc91-318f-405f-8807-25e5370f53ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:08.026786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585fcc91-318f-405f-8807-25e5370f53ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-585fcc91-318f-405f-8807-25e5370f53ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:58:08.031450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:08.032365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:08.033270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:08.037757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 522/1054] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:58:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:08.038994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588ecfb-31ee-4439-b3b2-f7d39c2b998e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:08.141078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6588ecfb-31ee-4439-b3b2-f7d39c2b998e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:08.176478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231674d9-d389-418e-afe4-607f3854f516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:08.177083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231674d9-d389-418e-afe4-607f3854f516 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:08.177383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231674d9-d389-418e-afe4-607f3854f516 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:08.178186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231674d9-d389-418e-afe4-607f3854f516 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:08.178995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231674d9-d389-418e-afe4-607f3854f516 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:08.179295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231674d9-d389-418e-afe4-607f3854f516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:08.180650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231674d9-d389-418e-afe4-607f3854f516 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:08.189132 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 533/1055] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:58:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:08.195037 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fcc91-318f-405f-8807-25e5370f53ed None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:08.212018 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:08.298169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585fcc91-318f-405f-8807-25e5370f53ed None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:08.332328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441e3b8-354d-45aa-bbf4-e8762b1c08f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:08.332948 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441e3b8-354d-45aa-bbf4-e8762b1c08f7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:08.333252 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441e3b8-354d-45aa-bbf4-e8762b1c08f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:08.333523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441e3b8-354d-45aa-bbf4-e8762b1c08f7 None admin] PATH_INFO: `/v3/users/5cc93020bec64689a2690e681e6ece9c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:08.347374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441e3b8-354d-45aa-bbf4-e8762b1c08f7 None admin] RBAC: Authorizing `identity:get_user(user_id=5cc93020bec64689a2690e681e6ece9c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:08.348817 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441e3b8-354d-45aa-bbf4-e8762b1c08f7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:08.353448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 523/1056] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:08 2022] GET /identity/v3/users/5cc93020bec64689a2690e681e6ece9c => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:08.356720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231674d9-d389-418e-afe4-607f3854f516 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:08.372416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:08.462319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231674d9-d389-418e-afe4-607f3854f516 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:08.498029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d0db8-224b-471c-9800-02104a26248d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:08.498635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d0db8-224b-471c-9800-02104a26248d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:08.498913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d0db8-224b-471c-9800-02104a26248d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:08.499185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d0db8-224b-471c-9800-02104a26248d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:08.546502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de3f27-3ea1-4ae4-b5e4-ee086a747905 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:08.547027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de3f27-3ea1-4ae4-b5e4-ee086a747905 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:08.547027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de3f27-3ea1-4ae4-b5e4-ee086a747905 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:08.548556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 524/1057] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:08.555685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9398e1-d1bf-43b6-a5be-da0bd2b8857c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:08.555977 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9398e1-d1bf-43b6-a5be-da0bd2b8857c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:08.556253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9398e1-d1bf-43b6-a5be-da0bd2b8857c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:08.572178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d0db8-224b-471c-9800-02104a26248d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:08.573700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d0db8-224b-471c-9800-02104a26248d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:08.575132 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6d9398e1-d1bf-43b6-a5be-da0bd2b8857c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:08.576363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6d9398e1-d1bf-43b6-a5be-da0bd2b8857c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:08.592449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:08.653626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 525/1058] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:58:08.681086 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce7162-3558-4406-b161-aa9647df47c1 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:08.681404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce7162-3558-4406-b161-aa9647df47c1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:08.681683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bce7162-3558-4406-b161-aa9647df47c1 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:08.682992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 526/1059] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:58:08.687710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd558d-209c-4b60-bd2e-ae7df842feae None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:08.687960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd558d-209c-4b60-bd2e-ae7df842feae None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:08.688205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd558d-209c-4b60-bd2e-ae7df842feae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:08.690558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 534/1060] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:58:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:58:08.694464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:08.696193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d0db8-224b-471c-9800-02104a26248d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:08.707057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0bdd558d-209c-4b60-bd2e-ae7df842feae None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:08.708226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0bdd558d-209c-4b60-bd2e-ae7df842feae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:08.780495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 527/1061] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:58:08.799749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d0db8-224b-471c-9800-02104a26248d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:08.813252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:08.835678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030b4c8-0165-4756-8292-1b24e1723841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:08.836363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b4c8-0165-4756-8292-1b24e1723841 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:08.836640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b4c8-0165-4756-8292-1b24e1723841 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:08.836907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b4c8-0165-4756-8292-1b24e1723841 None admin] PATH_INFO: `/v3/projects/0ba4b36f650846a197984252ffed93ec` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:08.838035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030b4c8-0165-4756-8292-1b24e1723841 None admin] RBAC: Authorizing `identity:get_project(project_id=0ba4b36f650846a197984252ffed93ec)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:08.839695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a030b4c8-0165-4756-8292-1b24e1723841 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:08.841501 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 535/1062] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:58:08 2022] GET /identity/v3/projects/0ba4b36f650846a197984252ffed93ec => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:08.847382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd558d-209c-4b60-bd2e-ae7df842feae None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:08.924569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030b4c8-0165-4756-8292-1b24e1723841 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:08.947984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd558d-209c-4b60-bd2e-ae7df842feae None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:08.978384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:08.984680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605c4bc-1d48-4daf-bf8b-d68365745910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:08.985306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605c4bc-1d48-4daf-bf8b-d68365745910 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:08.985582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605c4bc-1d48-4daf-bf8b-d68365745910 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:08.985852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605c4bc-1d48-4daf-bf8b-d68365745910 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:08.986563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605c4bc-1d48-4daf-bf8b-d68365745910 None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=0ba4b36f650846a197984252ffed93ec, user.id=5cc93020bec64689a2690e681e6ece9c) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:08.986869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605c4bc-1d48-4daf-bf8b-d68365745910 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:08.988287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605c4bc-1d48-4daf-bf8b-d68365745910 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:09.001152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 528/1063] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:58:08 2022] GET /identity/v3/role_assignments?user.id=5cc93020bec64689a2690e681e6ece9c&scope.project.id=0ba4b36f650846a197984252ffed93ec&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:09.041796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030b4c8-0165-4756-8292-1b24e1723841 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:09.045705 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605c4bc-1d48-4daf-bf8b-d68365745910 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:09.087861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de0ba6-2200-4739-8e6a-bd7977089fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:09.088758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de0ba6-2200-4739-8e6a-bd7977089fd3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:09.089034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de0ba6-2200-4739-8e6a-bd7977089fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:09.089343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de0ba6-2200-4739-8e6a-bd7977089fd3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:09.101051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8de0ba6-2200-4739-8e6a-bd7977089fd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e8de0ba6-2200-4739-8e6a-bd7977089fd3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:58:09.106001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:09.107163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:58:09.108129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:09.112552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 536/1064] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 22 21:58:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:58:09.113523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de0ba6-2200-4739-8e6a-bd7977089fd3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:09.163415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:09.172968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605c4bc-1d48-4daf-bf8b-d68365745910 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:09.212520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6151ec-83ce-4635-aa7d-1a9d83ffb74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:09.213413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6151ec-83ce-4635-aa7d-1a9d83ffb74c None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:09.213691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6151ec-83ce-4635-aa7d-1a9d83ffb74c None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:09.213959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6151ec-83ce-4635-aa7d-1a9d83ffb74c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:09.229434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8de0ba6-2200-4739-8e6a-bd7977089fd3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:09.268054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf41a7-d2df-4fb3-8ed8-ec4f2933180c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:09.268653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf41a7-d2df-4fb3-8ed8-ec4f2933180c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:09.268926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf41a7-d2df-4fb3-8ed8-ec4f2933180c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:09.269217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf41a7-d2df-4fb3-8ed8-ec4f2933180c None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:09.269872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf41a7-d2df-4fb3-8ed8-ec4f2933180c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:09.270169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf41a7-d2df-4fb3-8ed8-ec4f2933180c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:09.271736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf41a7-d2df-4fb3-8ed8-ec4f2933180c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:09.279843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 537/1065] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 22 21:58:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:09.285418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf41a7-d2df-4fb3-8ed8-ec4f2933180c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:09.287405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:09.294768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6151ec-83ce-4635-aa7d-1a9d83ffb74c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:09.296143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6151ec-83ce-4635-aa7d-1a9d83ffb74c None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:09.397024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 529/1066] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:58:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 21:58:09.398191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf41a7-d2df-4fb3-8ed8-ec4f2933180c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:09.402282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:09.403547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6151ec-83ce-4635-aa7d-1a9d83ffb74c None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:09.439305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8843e-619b-4edf-995f-cd093f9e95d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:09.440395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8843e-619b-4edf-995f-cd093f9e95d6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:09.440875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8843e-619b-4edf-995f-cd093f9e95d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:09.441403 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8843e-619b-4edf-995f-cd093f9e95d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:09.454177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8843e-619b-4edf-995f-cd093f9e95d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:09.455611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8843e-619b-4edf-995f-cd093f9e95d6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-3ae8843e-619b-4edf-995f-cd093f9e95d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:58:09.472025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:09.473032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:58:09.479858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:58:09.479858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:09.479858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:09.479858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 538/1067] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:58:09.479858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8843e-619b-4edf-995f-cd093f9e95d6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:09.499214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:09.505658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6151ec-83ce-4635-aa7d-1a9d83ffb74c None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:09.543842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf552cb4-22b7-49a5-a474-df3aef9634bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:09.544435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf552cb4-22b7-49a5-a474-df3aef9634bb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:09.544712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf552cb4-22b7-49a5-a474-df3aef9634bb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:09.545033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf552cb4-22b7-49a5-a474-df3aef9634bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:09.555720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf552cb4-22b7-49a5-a474-df3aef9634bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:09.557185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf552cb4-22b7-49a5-a474-df3aef9634bb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-bf552cb4-22b7-49a5-a474-df3aef9634bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:58:09.572413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:09.573456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:58:09.574333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:58:09.574333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:09.574333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:09.574333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 530/1068] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:09.579531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf552cb4-22b7-49a5-a474-df3aef9634bb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:09.589262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8843e-619b-4edf-995f-cd093f9e95d6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:09.627355 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74915843-7cfc-4b4a-8a3e-d307f528716a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:09.628654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74915843-7cfc-4b4a-8a3e-d307f528716a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:09.629069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74915843-7cfc-4b4a-8a3e-d307f528716a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:09.629504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74915843-7cfc-4b4a-8a3e-d307f528716a None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:09.630454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74915843-7cfc-4b4a-8a3e-d307f528716a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:09.631548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74915843-7cfc-4b4a-8a3e-d307f528716a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:09.633618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74915843-7cfc-4b4a-8a3e-d307f528716a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:09.649621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 539/1069] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:09.657295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74915843-7cfc-4b4a-8a3e-d307f528716a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:09.676515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:09.694988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf552cb4-22b7-49a5-a474-df3aef9634bb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:09.733613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09eea3f-12f9-45e3-ace0-176fee82b146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:09.734569 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09eea3f-12f9-45e3-ace0-176fee82b146 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:09.734849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09eea3f-12f9-45e3-ace0-176fee82b146 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:09.735122 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09eea3f-12f9-45e3-ace0-176fee82b146 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:09.735760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09eea3f-12f9-45e3-ace0-176fee82b146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:09.736203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09eea3f-12f9-45e3-ace0-176fee82b146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:09.737669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09eea3f-12f9-45e3-ace0-176fee82b146 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:09.750788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 531/1070] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:09.780340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74915843-7cfc-4b4a-8a3e-d307f528716a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:09.821838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4303728-d787-4a78-960c-0ffa4f54ee53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:09.822390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4303728-d787-4a78-960c-0ffa4f54ee53 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:09.822665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4303728-d787-4a78-960c-0ffa4f54ee53 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:09.822937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4303728-d787-4a78-960c-0ffa4f54ee53 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:09.831297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4303728-d787-4a78-960c-0ffa4f54ee53 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:09.832786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4303728-d787-4a78-960c-0ffa4f54ee53 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b4303728-d787-4a78-960c-0ffa4f54ee53 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:58:09.846327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:09.847343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:09.848286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:09.848286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:09.848286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:58:09.848286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:58:09.848286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:58:09.848286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:58:09.848286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:58:09.848286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:58:09.848286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:58:09.848286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:09.853466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 540/1071] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 22 21:58:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:58:09.854738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09eea3f-12f9-45e3-ace0-176fee82b146 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:09.867382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:09.970198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09eea3f-12f9-45e3-ace0-176fee82b146 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:10.008033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab103dd-a2f0-41e5-a457-b40156f1872f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:10.008799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab103dd-a2f0-41e5-a457-b40156f1872f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:10.009198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab103dd-a2f0-41e5-a457-b40156f1872f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:10.009561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab103dd-a2f0-41e5-a457-b40156f1872f None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:10.010398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab103dd-a2f0-41e5-a457-b40156f1872f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:10.010823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab103dd-a2f0-41e5-a457-b40156f1872f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:10.012325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab103dd-a2f0-41e5-a457-b40156f1872f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:10.029923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 532/1072] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 22 21:58:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:10.035458 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4303728-d787-4a78-960c-0ffa4f54ee53 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:10.124038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:10.149989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4303728-d787-4a78-960c-0ffa4f54ee53 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:10.183327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ca9e9-1c18-4eed-88cb-dd7b06a94e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:10.183950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ca9e9-1c18-4eed-88cb-dd7b06a94e47 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:10.184231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ca9e9-1c18-4eed-88cb-dd7b06a94e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:10.184504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8ca9e9-1c18-4eed-88cb-dd7b06a94e47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:10.195761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ca9e9-1c18-4eed-88cb-dd7b06a94e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:10.197215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8ca9e9-1c18-4eed-88cb-dd7b06a94e47 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-7d8ca9e9-1c18-4eed-88cb-dd7b06a94e47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:58:10.211950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:10.212919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:58:10.214555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:58:10.214555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:10.214555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:10.214555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 541/1073] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:58:10.218921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab103dd-a2f0-41e5-a457-b40156f1872f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:10.284159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:10.319557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab103dd-a2f0-41e5-a457-b40156f1872f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:10.353803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c4992-5cc0-4e24-8f95-f595d50f4935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:10.354400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c4992-5cc0-4e24-8f95-f595d50f4935 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:10.354679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c4992-5cc0-4e24-8f95-f595d50f4935 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:10.354952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c4992-5cc0-4e24-8f95-f595d50f4935 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:10.355581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c4992-5cc0-4e24-8f95-f595d50f4935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:10.355886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c4992-5cc0-4e24-8f95-f595d50f4935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:10.357290 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c4992-5cc0-4e24-8f95-f595d50f4935 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:10.368203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 533/1074] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:10.373953 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ca9e9-1c18-4eed-88cb-dd7b06a94e47 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:10.471022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8ca9e9-1c18-4eed-88cb-dd7b06a94e47 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:10.496326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:10.505192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc45c5-e5ee-4229-a811-0c5daf4fa3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:10.505835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc45c5-e5ee-4229-a811-0c5daf4fa3bc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:10.506104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc45c5-e5ee-4229-a811-0c5daf4fa3bc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:10.506371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc45c5-e5ee-4229-a811-0c5daf4fa3bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:10.512734 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc45c5-e5ee-4229-a811-0c5daf4fa3bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:10.514202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc45c5-e5ee-4229-a811-0c5daf4fa3bc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-70dc45c5-e5ee-4229-a811-0c5daf4fa3bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:58:10.525225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:10.526262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:10.527164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:58:10.527164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:58:10.527164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:58:10.527164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:58:10.527164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:58:10.527164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:58:10.527164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:58:10.527164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:10.527164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 542/1075] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:58:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:10.532128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c4992-5cc0-4e24-8f95-f595d50f4935 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:10.628476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c4992-5cc0-4e24-8f95-f595d50f4935 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:10.652566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:10.660965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea05475-1a1c-4eb5-a344-ddfba1c6b9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:10.661589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05475-1a1c-4eb5-a344-ddfba1c6b9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:10.662393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05475-1a1c-4eb5-a344-ddfba1c6b9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:10.662659 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea05475-1a1c-4eb5-a344-ddfba1c6b9d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:10.663335 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea05475-1a1c-4eb5-a344-ddfba1c6b9d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:10.663632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea05475-1a1c-4eb5-a344-ddfba1c6b9d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:10.665034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea05475-1a1c-4eb5-a344-ddfba1c6b9d7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:10.676543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 534/1076] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:58:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:10.683120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc45c5-e5ee-4229-a811-0c5daf4fa3bc None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:10.780276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc45c5-e5ee-4229-a811-0c5daf4fa3bc None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:10.813609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e63a82-c47f-4760-ba43-13471cedfacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:10.814256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e63a82-c47f-4760-ba43-13471cedfacc None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:10.814588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e63a82-c47f-4760-ba43-13471cedfacc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:10.814858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e63a82-c47f-4760-ba43-13471cedfacc None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/d011e54e12674eaeb82c68da628ecb9d/roles/f1c696334f084cccb674b2528a9fa65c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:10.816974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e63a82-c47f-4760-ba43-13471cedfacc None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=d011e54e12674eaeb82c68da628ecb9d, role_id=f1c696334f084cccb674b2528a9fa65c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:10.818579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e63a82-c47f-4760-ba43-13471cedfacc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:10.831113 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 543/1077] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:58:10 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/d011e54e12674eaeb82c68da628ecb9d/roles/f1c696334f084cccb674b2528a9fa65c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 21:58:11.162810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567714-2ce3-4b64-bc05-6ed1936b59ac None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:11.163090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567714-2ce3-4b64-bc05-6ed1936b59ac None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:11.163370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567714-2ce3-4b64-bc05-6ed1936b59ac None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:11.164718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 535/1078] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:11.170040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277cbcd-7977-4945-ae3e-7d12f8a25384 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:11.170324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277cbcd-7977-4945-ae3e-7d12f8a25384 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:11.170647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2277cbcd-7977-4945-ae3e-7d12f8a25384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:11.188937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2277cbcd-7977-4945-ae3e-7d12f8a25384 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:11.190154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2277cbcd-7977-4945-ae3e-7d12f8a25384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:11.233067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:11.258202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 544/1079] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:11.523484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74567714-2ce3-4b64-bc05-6ed1936b59ac None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:11.625823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74567714-2ce3-4b64-bc05-6ed1936b59ac None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:11.658250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5174d-26e2-47d0-8001-1cee1e52cf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:11.658870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5174d-26e2-47d0-8001-1cee1e52cf70 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:11.659122 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5174d-26e2-47d0-8001-1cee1e52cf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:11.659390 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd5174d-26e2-47d0-8001-1cee1e52cf70 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:11.669430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd5174d-26e2-47d0-8001-1cee1e52cf70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-5dd5174d-26e2-47d0-8001-1cee1e52cf70 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:58:11.674045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:11.675212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:11.676142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 536/1080] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:58:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:11.681641 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277cbcd-7977-4945-ae3e-7d12f8a25384 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:11.782189 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2277cbcd-7977-4945-ae3e-7d12f8a25384 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:11.814865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515c1bd-2c92-4311-a4af-eaa5c6895c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:11.815462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515c1bd-2c92-4311-a4af-eaa5c6895c35 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:11.815739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515c1bd-2c92-4311-a4af-eaa5c6895c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:11.816012 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6515c1bd-2c92-4311-a4af-eaa5c6895c35 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:11.816662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515c1bd-2c92-4311-a4af-eaa5c6895c35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:11.816966 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515c1bd-2c92-4311-a4af-eaa5c6895c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:11.818990 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6515c1bd-2c92-4311-a4af-eaa5c6895c35 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:11.826685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 545/1081] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:58:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:11.832488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5174d-26e2-47d0-8001-1cee1e52cf70 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:11.878123 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:11.931488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd5174d-26e2-47d0-8001-1cee1e52cf70 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:11.966275 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda34b92-8981-44c9-a6e0-3431a5d78cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:11.966896 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda34b92-8981-44c9-a6e0-3431a5d78cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:11.967172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda34b92-8981-44c9-a6e0-3431a5d78cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:11.967444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda34b92-8981-44c9-a6e0-3431a5d78cf5 None admin] PATH_INFO: `/v3/users/5cc93020bec64689a2690e681e6ece9c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:11.968547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda34b92-8981-44c9-a6e0-3431a5d78cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=5cc93020bec64689a2690e681e6ece9c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:11.969994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda34b92-8981-44c9-a6e0-3431a5d78cf5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:11.971564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 537/1082] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:11 2022] GET /identity/v3/users/5cc93020bec64689a2690e681e6ece9c => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:11.978119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515c1bd-2c92-4311-a4af-eaa5c6895c35 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:12.017410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:12.078861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6515c1bd-2c92-4311-a4af-eaa5c6895c35 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:12.112038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6733ebf3-e49d-40ed-9cb7-5667e2820e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:12.112652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733ebf3-e49d-40ed-9cb7-5667e2820e92 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:12.112925 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733ebf3-e49d-40ed-9cb7-5667e2820e92 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:12.113213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6733ebf3-e49d-40ed-9cb7-5667e2820e92 None admin] PATH_INFO: `/v3/projects/0ba4b36f650846a197984252ffed93ec` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:12.114304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733ebf3-e49d-40ed-9cb7-5667e2820e92 None admin] RBAC: Authorizing `identity:get_project(project_id=0ba4b36f650846a197984252ffed93ec)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:12.116254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6733ebf3-e49d-40ed-9cb7-5667e2820e92 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:12.117906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 546/1083] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:58:11 2022] GET /identity/v3/projects/0ba4b36f650846a197984252ffed93ec => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:12.123742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda34b92-8981-44c9-a6e0-3431a5d78cf5 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:12.218505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda34b92-8981-44c9-a6e0-3431a5d78cf5 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:12.232144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:12.252173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bbba73-9b1e-4b04-b957-6ced42c8b349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:12.252814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbba73-9b1e-4b04-b957-6ced42c8b349 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:12.253090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbba73-9b1e-4b04-b957-6ced42c8b349 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:12.253383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bbba73-9b1e-4b04-b957-6ced42c8b349 None admin] PATH_INFO: `/v3/projects/0ba4b36f650846a197984252ffed93ec/users/5cc93020bec64689a2690e681e6ece9c/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:12.255223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbba73-9b1e-4b04-b957-6ced42c8b349 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ba4b36f650846a197984252ffed93ec, user_id=5cc93020bec64689a2690e681e6ece9c, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:12.256729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bbba73-9b1e-4b04-b957-6ced42c8b349 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:12.268363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 538/1084] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:58:12 2022] PUT /identity/v3/projects/0ba4b36f650846a197984252ffed93ec/users/5cc93020bec64689a2690e681e6ece9c/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:58:12.986649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532bbca-7eb5-40dd-9516-9c5439ecb3e8 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:12.987206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532bbca-7eb5-40dd-9516-9c5439ecb3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:12.987206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c532bbca-7eb5-40dd-9516-9c5439ecb3e8 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:12.988515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 547/1085] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:12.993986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a818adc-cdf7-4d37-8059-10c0067287ad None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:12.994406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a818adc-cdf7-4d37-8059-10c0067287ad None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:12.994539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a818adc-cdf7-4d37-8059-10c0067287ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:13.012866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5a818adc-cdf7-4d37-8059-10c0067287ad None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:13.014087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5a818adc-cdf7-4d37-8059-10c0067287ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:13.079813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 539/1086] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:13.350023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532bbca-7eb5-40dd-9516-9c5439ecb3e8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:13.449073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c532bbca-7eb5-40dd-9516-9c5439ecb3e8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:13.455818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:13.482931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b398c5-2819-4c8c-8e85-9af3f01596da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:13.483549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b398c5-2819-4c8c-8e85-9af3f01596da None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:13.483821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b398c5-2819-4c8c-8e85-9af3f01596da None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:13.484088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b398c5-2819-4c8c-8e85-9af3f01596da None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:13.494532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b398c5-2819-4c8c-8e85-9af3f01596da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-23b398c5-2819-4c8c-8e85-9af3f01596da None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:58:13.499225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:13.500292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:13.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 548/1087] 10.222.0.25 () {56 vars in 1120 bytes} [Thu Sep 22 21:58:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:13.506324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a818adc-cdf7-4d37-8059-10c0067287ad None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:13.602543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a818adc-cdf7-4d37-8059-10c0067287ad None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:13.604722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:13.636546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8879d-2f42-4bdc-8e89-53002e52db4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:13.637163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8879d-2f42-4bdc-8e89-53002e52db4d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:13.637439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8879d-2f42-4bdc-8e89-53002e52db4d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:13.637706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8879d-2f42-4bdc-8e89-53002e52db4d None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:13.638378 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8879d-2f42-4bdc-8e89-53002e52db4d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:13.638675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8879d-2f42-4bdc-8e89-53002e52db4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:13.640019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8879d-2f42-4bdc-8e89-53002e52db4d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:13.647759 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 540/1088] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 22 21:58:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:13.653542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b398c5-2819-4c8c-8e85-9af3f01596da None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:13.751545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b398c5-2819-4c8c-8e85-9af3f01596da None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:13.783916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fa0d0-5179-4716-9b06-00b3e95e495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:13.784530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fa0d0-5179-4716-9b06-00b3e95e495d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:13.784803 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fa0d0-5179-4716-9b06-00b3e95e495d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:13.785069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fa0d0-5179-4716-9b06-00b3e95e495d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:13.795137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fa0d0-5179-4716-9b06-00b3e95e495d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:13.796560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fa0d0-5179-4716-9b06-00b3e95e495d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-2d1fa0d0-5179-4716-9b06-00b3e95e495d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:58:13.811548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:13.812735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:58:13.813861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:58:13.813861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:13.813861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:13.815235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 549/1089] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:13.821001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8879d-2f42-4bdc-8e89-53002e52db4d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:13.918969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8879d-2f42-4bdc-8e89-53002e52db4d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:13.952228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59600664-1edf-46ab-babc-af7266ce70ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:13.952830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59600664-1edf-46ab-babc-af7266ce70ce None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:13.953125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59600664-1edf-46ab-babc-af7266ce70ce None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:13.953413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59600664-1edf-46ab-babc-af7266ce70ce None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:13.954047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59600664-1edf-46ab-babc-af7266ce70ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:13.954347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59600664-1edf-46ab-babc-af7266ce70ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:13.955707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59600664-1edf-46ab-babc-af7266ce70ce None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:13.968357 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 541/1090] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:13.973935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fa0d0-5179-4716-9b06-00b3e95e495d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:13.994068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:14.079753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fa0d0-5179-4716-9b06-00b3e95e495d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:14.113495 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2794c-1f1b-46ec-baa9-bd7553dde05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:14.114114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2794c-1f1b-46ec-baa9-bd7553dde05b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:14.114390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2794c-1f1b-46ec-baa9-bd7553dde05b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:14.114688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d2794c-1f1b-46ec-baa9-bd7553dde05b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:14.122001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2794c-1f1b-46ec-baa9-bd7553dde05b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:14.123420 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d2794c-1f1b-46ec-baa9-bd7553dde05b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a3d2794c-1f1b-46ec-baa9-bd7553dde05b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 21:58:14.134444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:14.135562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:14.136549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:14.136549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:14.136549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 21:58:14.136549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 21:58:14.136549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 21:58:14.136549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 21:58:14.136549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 21:58:14.136549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 21:58:14.136549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 22 21:58:14.136549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:14.136549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 550/1091] 10.222.0.25 () {56 vars in 1093 bytes} [Thu Sep 22 21:58:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:14.141283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59600664-1edf-46ab-babc-af7266ce70ce None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:14.168638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:14.241516 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59600664-1edf-46ab-babc-af7266ce70ce None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:14.275778 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a4377-0a49-4e71-b726-baff4ec14671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:14.276384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a4377-0a49-4e71-b726-baff4ec14671 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:14.276664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a4377-0a49-4e71-b726-baff4ec14671 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:14.276935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a4377-0a49-4e71-b726-baff4ec14671 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:14.277693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a4377-0a49-4e71-b726-baff4ec14671 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:14.278001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a4377-0a49-4e71-b726-baff4ec14671 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:14.279414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a4377-0a49-4e71-b726-baff4ec14671 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:14.298464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 542/1092] 10.222.0.25 () {56 vars in 1133 bytes} [Thu Sep 22 21:58:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:14.304747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2794c-1f1b-46ec-baa9-bd7553dde05b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:14.387810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:14.409521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d2794c-1f1b-46ec-baa9-bd7553dde05b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:14.418963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150a018-6b03-4d17-a0ac-dfb894b7c9e9 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:14.419247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150a018-6b03-4d17-a0ac-dfb894b7c9e9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:14.419523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150a018-6b03-4d17-a0ac-dfb894b7c9e9 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:14.420854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 543/1093] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 21:58:14.425872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc398c-5f18-4c0a-bbdb-59ffb3c59dce None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:14.426156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc398c-5f18-4c0a-bbdb-59ffb3c59dce None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:14.426431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedc398c-5f18-4c0a-bbdb-59ffb3c59dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:14.444380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-dedc398c-5f18-4c0a-bbdb-59ffb3c59dce None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:14.445577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dedc398c-5f18-4c0a-bbdb-59ffb3c59dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:14.446730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb1fc9-7dfe-467d-ac0d-1c7967815f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:14.447331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb1fc9-7dfe-467d-ac0d-1c7967815f32 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:14.447606 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb1fc9-7dfe-467d-ac0d-1c7967815f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:14.447876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb1fc9-7dfe-467d-ac0d-1c7967815f32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:14.512170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 544/1094] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:58:14.518199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb1fc9-7dfe-467d-ac0d-1c7967815f32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:14.519639 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb1fc9-7dfe-467d-ac0d-1c7967815f32 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:14.593703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:14.616460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 551/1095] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:58:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:58:14.622262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc398c-5f18-4c0a-bbdb-59ffb3c59dce None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:14.637936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:14.721013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedc398c-5f18-4c0a-bbdb-59ffb3c59dce None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:14.754504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b338d4e-da8d-43a3-bc74-b99580b95d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:14.755128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b338d4e-da8d-43a3-bc74-b99580b95d12 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:14.755947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b338d4e-da8d-43a3-bc74-b99580b95d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:14.756218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b338d4e-da8d-43a3-bc74-b99580b95d12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:14.766093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b338d4e-da8d-43a3-bc74-b99580b95d12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:14.767461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b338d4e-da8d-43a3-bc74-b99580b95d12 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-0b338d4e-da8d-43a3-bc74-b99580b95d12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:58:14.781615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:14.782712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 21:58:14.783728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:58:14.783728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 21:58:14.783728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:14.783728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 545/1096] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:14.788423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb1fc9-7dfe-467d-ac0d-1c7967815f32 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:14.790565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b338d4e-da8d-43a3-bc74-b99580b95d12 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:14.855273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:14.887624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb1fc9-7dfe-467d-ac0d-1c7967815f32 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:14.890748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b338d4e-da8d-43a3-bc74-b99580b95d12 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:14.921277 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c4d2-52c7-4434-b625-cc2f551b13f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:14.922022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c4d2-52c7-4434-b625-cc2f551b13f7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:14.922283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c4d2-52c7-4434-b625-cc2f551b13f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:14.922555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170c4d2-52c7-4434-b625-cc2f551b13f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:14.923195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170c4d2-52c7-4434-b625-cc2f551b13f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:14.923496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170c4d2-52c7-4434-b625-cc2f551b13f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:14.924002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968fb2f-e490-46b5-92f9-a56114aa48a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:14.924866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7170c4d2-52c7-4434-b625-cc2f551b13f7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:14.925043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968fb2f-e490-46b5-92f9-a56114aa48a6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:14.925559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968fb2f-e490-46b5-92f9-a56114aa48a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:14.926046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968fb2f-e490-46b5-92f9-a56114aa48a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:14.937800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 552/1097] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 21:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:14.938796 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968fb2f-e490-46b5-92f9-a56114aa48a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:58:14.943807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c4d2-52c7-4434-b625-cc2f551b13f7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-a968fb2f-e490-46b5-92f9-a56114aa48a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:58:14.944108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:14.945250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:14.946461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:14.949405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 546/1098] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:58:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:58:14.950798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968fb2f-e490-46b5-92f9-a56114aa48a6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:14.988101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:15.048323 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170c4d2-52c7-4434-b625-cc2f551b13f7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:15.048936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968fb2f-e490-46b5-92f9-a56114aa48a6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:15.074790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:15.083973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac955a23-3770-4228-b836-0a69c8446f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:15.084490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b087ec6-a127-4d17-9f4d-7ee104de8414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:15.084623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac955a23-3770-4228-b836-0a69c8446f45 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:15.084838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac955a23-3770-4228-b836-0a69c8446f45 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:15.085124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac955a23-3770-4228-b836-0a69c8446f45 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:15.085216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b087ec6-a127-4d17-9f4d-7ee104de8414 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:15.085422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b087ec6-a127-4d17-9f4d-7ee104de8414 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:15.085691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b087ec6-a127-4d17-9f4d-7ee104de8414 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:15.085782 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac955a23-3770-4228-b836-0a69c8446f45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:15.086078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac955a23-3770-4228-b836-0a69c8446f45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:15.087441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac955a23-3770-4228-b836-0a69c8446f45 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:15.092457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b087ec6-a127-4d17-9f4d-7ee104de8414 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:15.093896 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b087ec6-a127-4d17-9f4d-7ee104de8414 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:15.096052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 547/1099] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:58:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:15.102522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac955a23-3770-4228-b836-0a69c8446f45 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-1b087ec6-a127-4d17-9f4d-7ee104de8414 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 21:58:15.105707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:15.106794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:15.107698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 21:58:15.107698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 21:58:15.107698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:58:15.107698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:58:15.107698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:58:15.107698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:58:15.107698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 21:58:15.107698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:15.107698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 553/1100] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 21:58:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 21:58:15.112487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b087ec6-a127-4d17-9f4d-7ee104de8414 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:15.203114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac955a23-3770-4228-b836-0a69c8446f45 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:15.206901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:15.211356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b087ec6-a127-4d17-9f4d-7ee104de8414 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:15.237810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a4815-d5ee-41d1-9d60-5a8ca00b7820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:15.238436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a4815-d5ee-41d1-9d60-5a8ca00b7820 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:15.238715 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a4815-d5ee-41d1-9d60-5a8ca00b7820 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:15.238985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a4815-d5ee-41d1-9d60-5a8ca00b7820 None admin] PATH_INFO: `/v3/users/5cc93020bec64689a2690e681e6ece9c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:15.240249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a4815-d5ee-41d1-9d60-5a8ca00b7820 None admin] RBAC: Authorizing `identity:get_user(user_id=5cc93020bec64689a2690e681e6ece9c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:15.241742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2a4815-d5ee-41d1-9d60-5a8ca00b7820 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:15.243653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 548/1101] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:15 2022] GET /identity/v3/users/5cc93020bec64689a2690e681e6ece9c => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:15.245790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b37069-41fc-4f39-923b-8446d263c5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:15.246400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b37069-41fc-4f39-923b-8446d263c5f1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:15.246678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b37069-41fc-4f39-923b-8446d263c5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:15.246950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b37069-41fc-4f39-923b-8446d263c5f1 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:15.247649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b37069-41fc-4f39-923b-8446d263c5f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:15.247960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b37069-41fc-4f39-923b-8446d263c5f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:15.249193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a4815-d5ee-41d1-9d60-5a8ca00b7820 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:15.249408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b37069-41fc-4f39-923b-8446d263c5f1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:15.263094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 554/1102] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 21:58:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:15.269049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b37069-41fc-4f39-923b-8446d263c5f1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:15.306161 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:15.350271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a4815-d5ee-41d1-9d60-5a8ca00b7820 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:15.368374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b37069-41fc-4f39-923b-8446d263c5f1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:15.384965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ebeaf-3008-4e4b-bd97-ab1ab05fe7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:15.385591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ebeaf-3008-4e4b-bd97-ab1ab05fe7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:15.385867 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ebeaf-3008-4e4b-bd97-ab1ab05fe7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:15.386138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ebeaf-3008-4e4b-bd97-ab1ab05fe7c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:15.403331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d01fa7-f4d0-4733-a4ad-cc464a57f8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:15.403923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d01fa7-f4d0-4733-a4ad-cc464a57f8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:15.404195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d01fa7-f4d0-4733-a4ad-cc464a57f8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:15.404455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d01fa7-f4d0-4733-a4ad-cc464a57f8f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:15.405188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d01fa7-f4d0-4733-a4ad-cc464a57f8f9 None admin] RBAC: Adding query filter params (role.id=f1c696334f084cccb674b2528a9fa65c, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=d011e54e12674eaeb82c68da628ecb9d) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:15.405489 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d01fa7-f4d0-4733-a4ad-cc464a57f8f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:15.406887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d01fa7-f4d0-4733-a4ad-cc464a57f8f9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:15.412553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 555/1103] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:58:15 2022] GET /identity/v3/role_assignments?user.id=d011e54e12674eaeb82c68da628ecb9d&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=f1c696334f084cccb674b2528a9fa65c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:15.455566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ebeaf-3008-4e4b-bd97-ab1ab05fe7c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:15.457019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ebeaf-3008-4e4b-bd97-ab1ab05fe7c3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:15.517435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:15.566364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 549/1104] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:58:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 21:58:15.572850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d01fa7-f4d0-4733-a4ad-cc464a57f8f9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:15.597810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:15.673338 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d01fa7-f4d0-4733-a4ad-cc464a57f8f9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:15.707777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a442e52-747e-46d0-9c3e-582dbc73e0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:15.708440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a442e52-747e-46d0-9c3e-582dbc73e0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:15.708712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a442e52-747e-46d0-9c3e-582dbc73e0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:15.708977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a442e52-747e-46d0-9c3e-582dbc73e0b3 None admin] PATH_INFO: `/v3/projects/0ba4b36f650846a197984252ffed93ec` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:15.710096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a442e52-747e-46d0-9c3e-582dbc73e0b3 None admin] RBAC: Authorizing `identity:get_project(project_id=0ba4b36f650846a197984252ffed93ec)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:15.711531 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a442e52-747e-46d0-9c3e-582dbc73e0b3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:15.713737 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 556/1105] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:58:15 2022] GET /identity/v3/projects/0ba4b36f650846a197984252ffed93ec => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:15.719797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ebeaf-3008-4e4b-bd97-ab1ab05fe7c3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:15.828283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ebeaf-3008-4e4b-bd97-ab1ab05fe7c3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:15.863513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bf001-7de6-4c17-8103-378e6e0db3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:15.864108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bf001-7de6-4c17-8103-378e6e0db3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:15.864394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bf001-7de6-4c17-8103-378e6e0db3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:15.865368 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bf001-7de6-4c17-8103-378e6e0db3a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:15.866159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bf001-7de6-4c17-8103-378e6e0db3a5 None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=0ba4b36f650846a197984252ffed93ec, user.id=5cc93020bec64689a2690e681e6ece9c) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:15.866466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bf001-7de6-4c17-8103-378e6e0db3a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:15.867886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22bf001-7de6-4c17-8103-378e6e0db3a5 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:15.873830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 550/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:58:15 2022] GET /identity/v3/role_assignments?user.id=5cc93020bec64689a2690e681e6ece9c&scope.project.id=0ba4b36f650846a197984252ffed93ec&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:18.068789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579c223-3b56-4d59-8761-f1e79c6a8a43 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:18.069268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579c223-3b56-4d59-8761-f1e79c6a8a43 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:18.069376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579c223-3b56-4d59-8761-f1e79c6a8a43 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:18.070703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 557/1107] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:18.075969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83453d0e-2e91-4d5b-92d9-c744831a2a61 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:18.076399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83453d0e-2e91-4d5b-92d9-c744831a2a61 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:18.076509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83453d0e-2e91-4d5b-92d9-c744831a2a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:18.095126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-83453d0e-2e91-4d5b-92d9-c744831a2a61 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:18.096316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-83453d0e-2e91-4d5b-92d9-c744831a2a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:18.110133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:18.161575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 551/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:18.428359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579c223-3b56-4d59-8761-f1e79c6a8a43 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:18.515373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:18.526294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579c223-3b56-4d59-8761-f1e79c6a8a43 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:18.558487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25428706-8f59-4c0c-8733-798907055e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:18.559087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25428706-8f59-4c0c-8733-798907055e2a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:18.559357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25428706-8f59-4c0c-8733-798907055e2a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:18.559627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25428706-8f59-4c0c-8733-798907055e2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:18.560700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25428706-8f59-4c0c-8733-798907055e2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:18.562126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25428706-8f59-4c0c-8733-798907055e2a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:18.563642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 558/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:18.569427 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83453d0e-2e91-4d5b-92d9-c744831a2a61 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:18.666320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83453d0e-2e91-4d5b-92d9-c744831a2a61 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:18.699471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc2146-d609-4760-94cd-a3376e0d34a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:18.699998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc2146-d609-4760-94cd-a3376e0d34a9 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:18.700276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc2146-d609-4760-94cd-a3376e0d34a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:18.700548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc2146-d609-4760-94cd-a3376e0d34a9 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:18.701222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc2146-d609-4760-94cd-a3376e0d34a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:18.702634 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc2146-d609-4760-94cd-a3376e0d34a9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:18.725839 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 552/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:58:18 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:58:20.902269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc67227-04de-4fbc-8207-56b03a5689ff None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:20.902806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc67227-04de-4fbc-8207-56b03a5689ff None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:20.902806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc67227-04de-4fbc-8207-56b03a5689ff None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:20.904138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 559/1111] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:20.909342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751e232-35f9-47db-bda5-4df2c7641f10 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:20.909714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751e232-35f9-47db-bda5-4df2c7641f10 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:20.909888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0751e232-35f9-47db-bda5-4df2c7641f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:20.922659 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:20.928629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0751e232-35f9-47db-bda5-4df2c7641f10 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:20.930226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0751e232-35f9-47db-bda5-4df2c7641f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:20.993935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 553/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:21.260844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc67227-04de-4fbc-8207-56b03a5689ff None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:21.358226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc67227-04de-4fbc-8207-56b03a5689ff None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:21.391234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6787b8-ce9c-4323-95be-ef5516926987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:21.391849 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6787b8-ce9c-4323-95be-ef5516926987 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:21.392125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6787b8-ce9c-4323-95be-ef5516926987 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:21.392396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6787b8-ce9c-4323-95be-ef5516926987 None admin] PATH_INFO: `/v3/roles/dea086603b04451680a14ae806205d79` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:21.393793 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6787b8-ce9c-4323-95be-ef5516926987 None admin] RBAC: Authorizing `identity:get_role(role_id=dea086603b04451680a14ae806205d79)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:21.395138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6787b8-ce9c-4323-95be-ef5516926987 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:21.396484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 560/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:21 2022] GET /identity/v3/roles/dea086603b04451680a14ae806205d79 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:21.402178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751e232-35f9-47db-bda5-4df2c7641f10 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:21.496982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0751e232-35f9-47db-bda5-4df2c7641f10 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:21.529949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac21095-deda-4f31-a1f8-41eebf936c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:21.530496 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac21095-deda-4f31-a1f8-41eebf936c92 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:21.530773 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac21095-deda-4f31-a1f8-41eebf936c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:21.531042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac21095-deda-4f31-a1f8-41eebf936c92 None admin] PATH_INFO: `/v3/users/1eeda38182ff4d9b91e9e7a2bc1606fd` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:21.533649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:21.544134 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac21095-deda-4f31-a1f8-41eebf936c92 None admin] RBAC: Authorizing `identity:get_user(user_id=1eeda38182ff4d9b91e9e7a2bc1606fd)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:21.545596 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac21095-deda-4f31-a1f8-41eebf936c92 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:21.547191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 554/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:21 2022] GET /identity/v3/users/1eeda38182ff4d9b91e9e7a2bc1606fd => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:21.553566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6787b8-ce9c-4323-95be-ef5516926987 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:21.580003 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:21.648506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6787b8-ce9c-4323-95be-ef5516926987 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:21.680636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d98d6-1f65-41a3-931b-7362be57b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:21.681245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d98d6-1f65-41a3-931b-7362be57b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:21.681520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d98d6-1f65-41a3-931b-7362be57b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:21.681788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87d98d6-1f65-41a3-931b-7362be57b2f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:21.746418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d98d6-1f65-41a3-931b-7362be57b2f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:21.747847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87d98d6-1f65-41a3-931b-7362be57b2f2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:21.772354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:21.842806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 561/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:58:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:58:21.849266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac21095-deda-4f31-a1f8-41eebf936c92 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:21.944112 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac21095-deda-4f31-a1f8-41eebf936c92 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:21.976669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3a3e8-12ba-4ada-ae1d-0dbdd698ede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:21.977302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3a3e8-12ba-4ada-ae1d-0dbdd698ede9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:21.977579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3a3e8-12ba-4ada-ae1d-0dbdd698ede9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:21.977848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3a3e8-12ba-4ada-ae1d-0dbdd698ede9 None admin] PATH_INFO: `/v3/projects/0ba4b36f650846a197984252ffed93ec` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:21.978935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3a3e8-12ba-4ada-ae1d-0dbdd698ede9 None admin] RBAC: Authorizing `identity:get_project(project_id=0ba4b36f650846a197984252ffed93ec)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:21.980338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3a3e8-12ba-4ada-ae1d-0dbdd698ede9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:21.981944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 555/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:58:21 2022] GET /identity/v3/projects/0ba4b36f650846a197984252ffed93ec => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:21.988122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d98d6-1f65-41a3-931b-7362be57b2f2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:22.084769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87d98d6-1f65-41a3-931b-7362be57b2f2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:22.109922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:22.118888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d51570-a56a-4157-86b7-586327bc375f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:22.119480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d51570-a56a-4157-86b7-586327bc375f None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:22.119755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d51570-a56a-4157-86b7-586327bc375f None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:22.120025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d51570-a56a-4157-86b7-586327bc375f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:22.120731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d51570-a56a-4157-86b7-586327bc375f None admin] RBAC: Adding query filter params (role.id=dea086603b04451680a14ae806205d79, scope.project.id=0ba4b36f650846a197984252ffed93ec, user.id=1eeda38182ff4d9b91e9e7a2bc1606fd) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:22.121036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d51570-a56a-4157-86b7-586327bc375f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:22.122469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d51570-a56a-4157-86b7-586327bc375f None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:22.127610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 562/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:58:21 2022] GET /identity/v3/role_assignments?user.id=1eeda38182ff4d9b91e9e7a2bc1606fd&scope.project.id=0ba4b36f650846a197984252ffed93ec&role.id=dea086603b04451680a14ae806205d79 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:24.245855 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6036f-8669-4317-b401-431b8628968b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:24.246417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6036f-8669-4317-b401-431b8628968b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:24.246417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6036f-8669-4317-b401-431b8628968b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:24.247722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 556/1118] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:24.252772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6b04-4d83-4d5d-a096-702b371bcb0c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:24.253218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6b04-4d83-4d5d-a096-702b371bcb0c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:24.253338 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b6b04-4d83-4d5d-a096-702b371bcb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:24.271997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f06b6b04-4d83-4d5d-a096-702b371bcb0c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:24.273197 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f06b6b04-4d83-4d5d-a096-702b371bcb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:24.336366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 563/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:24.605338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6036f-8669-4317-b401-431b8628968b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:24.665442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:24.708315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6036f-8669-4317-b401-431b8628968b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:24.752644 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81bc20-079d-444e-be52-dce4f693e22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:24.753286 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81bc20-079d-444e-be52-dce4f693e22f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:24.753556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81bc20-079d-444e-be52-dce4f693e22f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:24.753820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de81bc20-079d-444e-be52-dce4f693e22f None admin] PATH_INFO: `/v3/roles/dea086603b04451680a14ae806205d79` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:24.755307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81bc20-079d-444e-be52-dce4f693e22f None admin] RBAC: Authorizing `identity:get_role(role_id=dea086603b04451680a14ae806205d79)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:24.756907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de81bc20-079d-444e-be52-dce4f693e22f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:24.759128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 557/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:24 2022] GET /identity/v3/roles/dea086603b04451680a14ae806205d79 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:24.764764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b6b04-4d83-4d5d-a096-702b371bcb0c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:24.860123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b6b04-4d83-4d5d-a096-702b371bcb0c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:24.879382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:24.893141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69d8a3-85c2-4269-8434-fe08b5a9c8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:24.893757 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69d8a3-85c2-4269-8434-fe08b5a9c8ba None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:24.894033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69d8a3-85c2-4269-8434-fe08b5a9c8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:24.894303 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69d8a3-85c2-4269-8434-fe08b5a9c8ba None admin] PATH_INFO: `/v3/users/1eeda38182ff4d9b91e9e7a2bc1606fd` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:24.895610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69d8a3-85c2-4269-8434-fe08b5a9c8ba None admin] RBAC: Authorizing `identity:get_user(user_id=1eeda38182ff4d9b91e9e7a2bc1606fd)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:24.897029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f69d8a3-85c2-4269-8434-fe08b5a9c8ba None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:24.898611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 564/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:24 2022] GET /identity/v3/users/1eeda38182ff4d9b91e9e7a2bc1606fd => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:24.904364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81bc20-079d-444e-be52-dce4f693e22f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:25.001132 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de81bc20-079d-444e-be52-dce4f693e22f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:25.027859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:25.033829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fa97d-7d9b-425c-9456-7762e097d134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:25.034441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa97d-7d9b-425c-9456-7762e097d134 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:25.034717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa97d-7d9b-425c-9456-7762e097d134 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:25.034989 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa97d-7d9b-425c-9456-7762e097d134 None admin] PATH_INFO: `/v3/projects/0ba4b36f650846a197984252ffed93ec` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:25.036599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fa97d-7d9b-425c-9456-7762e097d134 None admin] RBAC: Authorizing `identity:get_project(project_id=0ba4b36f650846a197984252ffed93ec)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:25.038045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fa97d-7d9b-425c-9456-7762e097d134 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:25.039646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 558/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:58:24 2022] GET /identity/v3/projects/0ba4b36f650846a197984252ffed93ec => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:25.045588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69d8a3-85c2-4269-8434-fe08b5a9c8ba None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:25.143150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69d8a3-85c2-4269-8434-fe08b5a9c8ba None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:25.176221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fbefc-e2e8-4f0d-aa0b-da83997c88d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:25.176963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fbefc-e2e8-4f0d-aa0b-da83997c88d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:25.177265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fbefc-e2e8-4f0d-aa0b-da83997c88d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:25.177542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fbefc-e2e8-4f0d-aa0b-da83997c88d8 None admin] PATH_INFO: `/v3/projects/0ba4b36f650846a197984252ffed93ec/users/1eeda38182ff4d9b91e9e7a2bc1606fd/roles/dea086603b04451680a14ae806205d79` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:25.179655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fbefc-e2e8-4f0d-aa0b-da83997c88d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0ba4b36f650846a197984252ffed93ec, user_id=1eeda38182ff4d9b91e9e7a2bc1606fd, role_id=dea086603b04451680a14ae806205d79)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:25.181072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fbefc-e2e8-4f0d-aa0b-da83997c88d8 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:25.193129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 565/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:58:25 2022] PUT /identity/v3/projects/0ba4b36f650846a197984252ffed93ec/users/1eeda38182ff4d9b91e9e7a2bc1606fd/roles/dea086603b04451680a14ae806205d79 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:58:27.302785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe1e58-2b65-47e5-91ac-69328acd3620 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:27.303343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe1e58-2b65-47e5-91ac-69328acd3620 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:27.303343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe1e58-2b65-47e5-91ac-69328acd3620 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:27.304673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 559/1124] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:27.310099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98323886-f380-4cb3-8aad-7500b1e7d125 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:27.310496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98323886-f380-4cb3-8aad-7500b1e7d125 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:27.310599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98323886-f380-4cb3-8aad-7500b1e7d125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:27.328178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-98323886-f380-4cb3-8aad-7500b1e7d125 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:27.329386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-98323886-f380-4cb3-8aad-7500b1e7d125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:27.371895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:27.394520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 566/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:27.663030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe1e58-2b65-47e5-91ac-69328acd3620 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:27.762546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe1e58-2b65-47e5-91ac-69328acd3620 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:27.800875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cf2a0-5452-4762-be36-fc08a4ae732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:27.801550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cf2a0-5452-4762-be36-fc08a4ae732d None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:27.801827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cf2a0-5452-4762-be36-fc08a4ae732d None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:27.802101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cf2a0-5452-4762-be36-fc08a4ae732d None admin] PATH_INFO: `/v3/roles/dea086603b04451680a14ae806205d79` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:27.803489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cf2a0-5452-4762-be36-fc08a4ae732d None admin] RBAC: Authorizing `identity:get_role(role_id=dea086603b04451680a14ae806205d79)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:27.804847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cf2a0-5452-4762-be36-fc08a4ae732d None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:27.806201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 560/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:27 2022] GET /identity/v3/roles/dea086603b04451680a14ae806205d79 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:27.811991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98323886-f380-4cb3-8aad-7500b1e7d125 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:27.907934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98323886-f380-4cb3-8aad-7500b1e7d125 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:27.939755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b8ce0-a6db-430a-a193-0ffc7e83d82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:27.940307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b8ce0-a6db-430a-a193-0ffc7e83d82b None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:27.940584 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b8ce0-a6db-430a-a193-0ffc7e83d82b None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:27.940853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b8ce0-a6db-430a-a193-0ffc7e83d82b None admin] PATH_INFO: `/v3/users/1eeda38182ff4d9b91e9e7a2bc1606fd` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:27.942613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b8ce0-a6db-430a-a193-0ffc7e83d82b None admin] RBAC: Authorizing `identity:get_user(user_id=1eeda38182ff4d9b91e9e7a2bc1606fd)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:27.944147 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b8ce0-a6db-430a-a193-0ffc7e83d82b None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:27.945730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 567/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:27 2022] GET /identity/v3/users/1eeda38182ff4d9b91e9e7a2bc1606fd => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:27.951428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cf2a0-5452-4762-be36-fc08a4ae732d None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:28.014745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:28.048459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cf2a0-5452-4762-be36-fc08a4ae732d None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:28.080551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ab369-f887-44aa-add1-b364143eef2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:28.081170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ab369-f887-44aa-add1-b364143eef2a None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:28.081446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ab369-f887-44aa-add1-b364143eef2a None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:28.081710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ab369-f887-44aa-add1-b364143eef2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:28.146445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ab369-f887-44aa-add1-b364143eef2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:28.147861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7ab369-f887-44aa-add1-b364143eef2a None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:28.207186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:28.242399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 561/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:58:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:58:28.248447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b8ce0-a6db-430a-a193-0ffc7e83d82b None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:28.294178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:28.345031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b8ce0-a6db-430a-a193-0ffc7e83d82b None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:28.377505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091abcd2-ffaf-495f-bbdc-2b24c63776ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:28.378111 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091abcd2-ffaf-495f-bbdc-2b24c63776ba None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:28.378385 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091abcd2-ffaf-495f-bbdc-2b24c63776ba None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:28.378652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091abcd2-ffaf-495f-bbdc-2b24c63776ba None admin] PATH_INFO: `/v3/projects/0ba4b36f650846a197984252ffed93ec` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:28.379919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091abcd2-ffaf-495f-bbdc-2b24c63776ba None admin] RBAC: Authorizing `identity:get_project(project_id=0ba4b36f650846a197984252ffed93ec)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:28.381357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091abcd2-ffaf-495f-bbdc-2b24c63776ba None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:28.382948 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 568/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:58:28 2022] GET /identity/v3/projects/0ba4b36f650846a197984252ffed93ec => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:28.389052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ab369-f887-44aa-add1-b364143eef2a None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:28.485797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7ab369-f887-44aa-add1-b364143eef2a None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:28.518792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbc88c-9842-44b3-9076-0978f4ee5bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:28.519377 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc88c-9842-44b3-9076-0978f4ee5bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:28.519653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc88c-9842-44b3-9076-0978f4ee5bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:28.520508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc88c-9842-44b3-9076-0978f4ee5bf8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:28.521513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc88c-9842-44b3-9076-0978f4ee5bf8 None admin] RBAC: Adding query filter params (role.id=dea086603b04451680a14ae806205d79, scope.project.id=0ba4b36f650846a197984252ffed93ec, user.id=1eeda38182ff4d9b91e9e7a2bc1606fd) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:28.521831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc88c-9842-44b3-9076-0978f4ee5bf8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:28.523235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc88c-9842-44b3-9076-0978f4ee5bf8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:28.528806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 562/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:58:28 2022] GET /identity/v3/role_assignments?user.id=1eeda38182ff4d9b91e9e7a2bc1606fd&scope.project.id=0ba4b36f650846a197984252ffed93ec&role.id=dea086603b04451680a14ae806205d79 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:30.751679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e737-c000-43ef-99a8-629b5f30e392 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:30.752209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e737-c000-43ef-99a8-629b5f30e392 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:30.752301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524e737-c000-43ef-99a8-629b5f30e392 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:30.753713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 569/1131] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:30.759507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6f9b9-caec-4ea3-b5ad-ebbebfa4bed0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:30.759941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6f9b9-caec-4ea3-b5ad-ebbebfa4bed0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:30.760035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd6f9b9-caec-4ea3-b5ad-ebbebfa4bed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:30.778354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3cd6f9b9-caec-4ea3-b5ad-ebbebfa4bed0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:30.779524 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3cd6f9b9-caec-4ea3-b5ad-ebbebfa4bed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:30.803312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:30.845476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 563/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:31.119360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524e737-c000-43ef-99a8-629b5f30e392 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:31.218373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524e737-c000-43ef-99a8-629b5f30e392 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:31.232472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:31.252010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f4ba6d-beab-4235-8f3b-8750a5598afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:31.252626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4ba6d-beab-4235-8f3b-8750a5598afc None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:31.252902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4ba6d-beab-4235-8f3b-8750a5598afc None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:31.253202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f4ba6d-beab-4235-8f3b-8750a5598afc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:31.254484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4ba6d-beab-4235-8f3b-8750a5598afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:31.255908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f4ba6d-beab-4235-8f3b-8750a5598afc None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:31.257495 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 570/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:31.263662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6f9b9-caec-4ea3-b5ad-ebbebfa4bed0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:31.365174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd6f9b9-caec-4ea3-b5ad-ebbebfa4bed0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:31.397815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669627a7-db6e-4923-ba41-2a8d5a2b943b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:31.398400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669627a7-db6e-4923-ba41-2a8d5a2b943b None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:31.398672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669627a7-db6e-4923-ba41-2a8d5a2b943b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:31.398938 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669627a7-db6e-4923-ba41-2a8d5a2b943b None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:31.399570 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669627a7-db6e-4923-ba41-2a8d5a2b943b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:31.400968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669627a7-db6e-4923-ba41-2a8d5a2b943b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:31.426566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 564/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:58:31 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:58:33.589470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445485b4-83e0-4097-92cf-c5a64c33283d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:33.590042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445485b4-83e0-4097-92cf-c5a64c33283d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:33.590042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445485b4-83e0-4097-92cf-c5a64c33283d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:33.591339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 571/1135] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:33.596481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e0a8a-2b7f-436b-abf1-0a5400901ff1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:33.596901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e0a8a-2b7f-436b-abf1-0a5400901ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:33.597025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e0a8a-2b7f-436b-abf1-0a5400901ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:33.615296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-767e0a8a-2b7f-436b-abf1-0a5400901ff1 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:33.616463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-767e0a8a-2b7f-436b-abf1-0a5400901ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:33.623546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:33.681191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 565/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:33.948160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445485b4-83e0-4097-92cf-c5a64c33283d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:34.044968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445485b4-83e0-4097-92cf-c5a64c33283d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:34.078967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fa059-0ca7-493f-8e53-c1aee681cbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:34.079588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa059-0ca7-493f-8e53-c1aee681cbbf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:34.079862 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa059-0ca7-493f-8e53-c1aee681cbbf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:34.080129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fa059-0ca7-493f-8e53-c1aee681cbbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:34.081247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fa059-0ca7-493f-8e53-c1aee681cbbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:34.082685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fa059-0ca7-493f-8e53-c1aee681cbbf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:34.084235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 572/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 21:58:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:34.090145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e0a8a-2b7f-436b-abf1-0a5400901ff1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:34.184331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e0a8a-2b7f-436b-abf1-0a5400901ff1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:34.217172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e5f09-dd97-4047-b5ab-030d766a2f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:34.217746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e5f09-dd97-4047-b5ab-030d766a2f04 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:34.218020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e5f09-dd97-4047-b5ab-030d766a2f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:34.218289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e5f09-dd97-4047-b5ab-030d766a2f04 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:34.218927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e5f09-dd97-4047-b5ab-030d766a2f04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:34.220325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e5f09-dd97-4047-b5ab-030d766a2f04 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:34.236313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:34.244159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 566/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:58:34 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:58:36.404548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430bcaa-1cd9-4d8a-9cb1-78e603a51593 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:36.405047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430bcaa-1cd9-4d8a-9cb1-78e603a51593 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:36.405047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6430bcaa-1cd9-4d8a-9cb1-78e603a51593 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:36.406368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 573/1139] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:36.412409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d1950-d9bf-4722-baf3-675a63126227 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:36.412799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d1950-d9bf-4722-baf3-675a63126227 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:36.412950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d1950-d9bf-4722-baf3-675a63126227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:36.430736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4f9d1950-d9bf-4722-baf3-675a63126227 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:36.431939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4f9d1950-d9bf-4722-baf3-675a63126227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:36.495597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 567/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:36.767305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430bcaa-1cd9-4d8a-9cb1-78e603a51593 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:36.818566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:36.868558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6430bcaa-1cd9-4d8a-9cb1-78e603a51593 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:36.900167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d4f86-e364-47c6-bca7-576cd60ce414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:36.900768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d4f86-e364-47c6-bca7-576cd60ce414 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:36.901041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d4f86-e364-47c6-bca7-576cd60ce414 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:36.901336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d4f86-e364-47c6-bca7-576cd60ce414 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:36.911360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d4f86-e364-47c6-bca7-576cd60ce414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-cb7d4f86-e364-47c6-bca7-576cd60ce414 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:58:36.916016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:36.917081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:36.918154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 574/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:58:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:36.923125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d1950-d9bf-4722-baf3-675a63126227 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:37.017925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d1950-d9bf-4722-baf3-675a63126227 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:37.043149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:37.051282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f0d60-f92f-46e5-820f-e2657961e276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:37.051888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f0d60-f92f-46e5-820f-e2657961e276 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:37.052162 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f0d60-f92f-46e5-820f-e2657961e276 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:37.052433 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f0d60-f92f-46e5-820f-e2657961e276 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:37.053092 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f0d60-f92f-46e5-820f-e2657961e276 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:37.053426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f0d60-f92f-46e5-820f-e2657961e276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:37.054786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f0d60-f92f-46e5-820f-e2657961e276 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:37.063043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 568/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:58:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:37.068955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d4f86-e364-47c6-bca7-576cd60ce414 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:37.166545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d4f86-e364-47c6-bca7-576cd60ce414 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:37.176835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:37.199372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913f733-dfc1-4bbd-b4dc-bc3557f7f2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:37.199983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913f733-dfc1-4bbd-b4dc-bc3557f7f2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:37.200259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913f733-dfc1-4bbd-b4dc-bc3557f7f2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:37.200530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c913f733-dfc1-4bbd-b4dc-bc3557f7f2c0 None admin] PATH_INFO: `/v3/users/dfe1dbba21fe48ac90a8966e49a46b0f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:37.213870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913f733-dfc1-4bbd-b4dc-bc3557f7f2c0 None admin] RBAC: Authorizing `identity:get_user(user_id=dfe1dbba21fe48ac90a8966e49a46b0f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:37.215317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c913f733-dfc1-4bbd-b4dc-bc3557f7f2c0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:37.216885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 575/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:37 2022] GET /identity/v3/users/dfe1dbba21fe48ac90a8966e49a46b0f => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:37.223417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f0d60-f92f-46e5-820f-e2657961e276 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:37.319837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f0d60-f92f-46e5-820f-e2657961e276 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:37.352713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9841a7-60be-448d-94af-d544e11868fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:37.353330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9841a7-60be-448d-94af-d544e11868fc None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:37.353612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9841a7-60be-448d-94af-d544e11868fc None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:37.353886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9841a7-60be-448d-94af-d544e11868fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:37.410020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:37.420491 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9841a7-60be-448d-94af-d544e11868fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:37.421954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9841a7-60be-448d-94af-d544e11868fc None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:37.517887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 569/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:58:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 21:58:37.523913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913f733-dfc1-4bbd-b4dc-bc3557f7f2c0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:37.620059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c913f733-dfc1-4bbd-b4dc-bc3557f7f2c0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:37.651802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94950b-1e2a-48cb-8cb9-d2dac0be856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:37.652412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94950b-1e2a-48cb-8cb9-d2dac0be856d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:37.652688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94950b-1e2a-48cb-8cb9-d2dac0be856d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:37.652958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94950b-1e2a-48cb-8cb9-d2dac0be856d None admin] PATH_INFO: `/v3/projects/3bb2b67de0324cda962eeb67a6416e9c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:37.654087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94950b-1e2a-48cb-8cb9-d2dac0be856d None admin] RBAC: Authorizing `identity:get_project(project_id=3bb2b67de0324cda962eeb67a6416e9c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:37.655493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94950b-1e2a-48cb-8cb9-d2dac0be856d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:37.657085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 576/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:58:37 2022] GET /identity/v3/projects/3bb2b67de0324cda962eeb67a6416e9c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:37.663639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9841a7-60be-448d-94af-d544e11868fc None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:37.750902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:37.760976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9841a7-60be-448d-94af-d544e11868fc None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:37.794534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ee9f1-90de-4041-b747-05188dbae05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:37.795124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee9f1-90de-4041-b747-05188dbae05e None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:37.795402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee9f1-90de-4041-b747-05188dbae05e None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:37.795672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ee9f1-90de-4041-b747-05188dbae05e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:37.796382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ee9f1-90de-4041-b747-05188dbae05e None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=3bb2b67de0324cda962eeb67a6416e9c, user.id=dfe1dbba21fe48ac90a8966e49a46b0f) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:37.796693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ee9f1-90de-4041-b747-05188dbae05e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:37.798140 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ee9f1-90de-4041-b747-05188dbae05e None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:37.803921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 570/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:58:37 2022] GET /identity/v3/role_assignments?user.id=dfe1dbba21fe48ac90a8966e49a46b0f&scope.project.id=3bb2b67de0324cda962eeb67a6416e9c&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:39.929943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297dd7d-7d9e-4803-8ce0-5ed7ab8d3eb3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:39.930502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297dd7d-7d9e-4803-8ce0-5ed7ab8d3eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:39.930502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297dd7d-7d9e-4803-8ce0-5ed7ab8d3eb3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:39.931827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 577/1147] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:39.936942 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826c9c4-eae1-4109-8d2d-f910becee230 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:39.937357 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826c9c4-eae1-4109-8d2d-f910becee230 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:39.937517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826c9c4-eae1-4109-8d2d-f910becee230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:39.955658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7826c9c4-eae1-4109-8d2d-f910becee230 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:39.957512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7826c9c4-eae1-4109-8d2d-f910becee230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:40.021579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 571/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:40.290950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297dd7d-7d9e-4803-8ce0-5ed7ab8d3eb3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:40.326312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:40.389370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297dd7d-7d9e-4803-8ce0-5ed7ab8d3eb3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:40.430286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbeb0f-97db-44bc-a7e6-0dc38b3297e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:40.430923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbeb0f-97db-44bc-a7e6-0dc38b3297e6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:40.431199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbeb0f-97db-44bc-a7e6-0dc38b3297e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:40.431464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cbeb0f-97db-44bc-a7e6-0dc38b3297e6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:40.442137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cbeb0f-97db-44bc-a7e6-0dc38b3297e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-f7cbeb0f-97db-44bc-a7e6-0dc38b3297e6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:58:40.446889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:40.448030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:40.449087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 578/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:58:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:40.454183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826c9c4-eae1-4109-8d2d-f910becee230 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:40.534461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:40.551319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826c9c4-eae1-4109-8d2d-f910becee230 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:40.583023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875283cc-c41d-437a-85e6-0ba8c33ff752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:40.583612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875283cc-c41d-437a-85e6-0ba8c33ff752 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:40.583886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875283cc-c41d-437a-85e6-0ba8c33ff752 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:40.584153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875283cc-c41d-437a-85e6-0ba8c33ff752 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:40.584795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875283cc-c41d-437a-85e6-0ba8c33ff752 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:40.585090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875283cc-c41d-437a-85e6-0ba8c33ff752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:40.586469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875283cc-c41d-437a-85e6-0ba8c33ff752 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:40.594410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 572/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:58:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:40.600807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbeb0f-97db-44bc-a7e6-0dc38b3297e6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:40.691245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:40.697332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cbeb0f-97db-44bc-a7e6-0dc38b3297e6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:40.732503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575da6e2-010d-490e-ab91-f3266b52fab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:40.733138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575da6e2-010d-490e-ab91-f3266b52fab4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:40.733419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575da6e2-010d-490e-ab91-f3266b52fab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:40.733688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575da6e2-010d-490e-ab91-f3266b52fab4 None admin] PATH_INFO: `/v3/users/dfe1dbba21fe48ac90a8966e49a46b0f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:40.734779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575da6e2-010d-490e-ab91-f3266b52fab4 None admin] RBAC: Authorizing `identity:get_user(user_id=dfe1dbba21fe48ac90a8966e49a46b0f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:40.736182 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575da6e2-010d-490e-ab91-f3266b52fab4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:40.738360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 579/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:40 2022] GET /identity/v3/users/dfe1dbba21fe48ac90a8966e49a46b0f => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:40.744193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875283cc-c41d-437a-85e6-0ba8c33ff752 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:40.840988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875283cc-c41d-437a-85e6-0ba8c33ff752 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:40.873779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534c5d95-5cf6-44de-a844-20502ddbd3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:40.874387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c5d95-5cf6-44de-a844-20502ddbd3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:40.874665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c5d95-5cf6-44de-a844-20502ddbd3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:40.874937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534c5d95-5cf6-44de-a844-20502ddbd3f0 None admin] PATH_INFO: `/v3/projects/3bb2b67de0324cda962eeb67a6416e9c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:40.876041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c5d95-5cf6-44de-a844-20502ddbd3f0 None admin] RBAC: Authorizing `identity:get_project(project_id=3bb2b67de0324cda962eeb67a6416e9c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:40.877484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534c5d95-5cf6-44de-a844-20502ddbd3f0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:40.879118 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 573/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:58:40 2022] GET /identity/v3/projects/3bb2b67de0324cda962eeb67a6416e9c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:40.884898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575da6e2-010d-490e-ab91-f3266b52fab4 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:40.984378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575da6e2-010d-490e-ab91-f3266b52fab4 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:41.017652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a08e0-d681-4dd4-9054-048fd434d1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:41.018288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a08e0-d681-4dd4-9054-048fd434d1fa None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:41.018567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a08e0-d681-4dd4-9054-048fd434d1fa None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:41.018840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a08e0-d681-4dd4-9054-048fd434d1fa None admin] PATH_INFO: `/v3/projects/3bb2b67de0324cda962eeb67a6416e9c/users/dfe1dbba21fe48ac90a8966e49a46b0f/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:41.021678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a08e0-d681-4dd4-9054-048fd434d1fa None admin] RBAC: Authorizing `identity:create_grant(project_id=3bb2b67de0324cda962eeb67a6416e9c, user_id=dfe1dbba21fe48ac90a8966e49a46b0f, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:41.023086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a08e0-d681-4dd4-9054-048fd434d1fa None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:41.036444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 580/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:58:40 2022] PUT /identity/v3/projects/3bb2b67de0324cda962eeb67a6416e9c/users/dfe1dbba21fe48ac90a8966e49a46b0f/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:58:43.254946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960118d-1b47-4ba4-b48d-51f87a95d27f None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:43.255520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960118d-1b47-4ba4-b48d-51f87a95d27f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:43.255520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960118d-1b47-4ba4-b48d-51f87a95d27f None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:43.256803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 574/1154] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:43.262060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1849e-4c8a-4f91-8b95-69e05c23e2b9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:43.262507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1849e-4c8a-4f91-8b95-69e05c23e2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:43.262608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc1849e-4c8a-4f91-8b95-69e05c23e2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:43.280210 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-afc1849e-4c8a-4f91-8b95-69e05c23e2b9 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:43.281406 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-afc1849e-4c8a-4f91-8b95-69e05c23e2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:43.297969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:43.347817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 581/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:58:43.616857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960118d-1b47-4ba4-b48d-51f87a95d27f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:43.632509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:43.715897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960118d-1b47-4ba4-b48d-51f87a95d27f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:43.749270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d972e1-166b-4105-8ed3-838cd41978eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:43.749877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d972e1-166b-4105-8ed3-838cd41978eb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:43.750152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d972e1-166b-4105-8ed3-838cd41978eb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:43.750421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d972e1-166b-4105-8ed3-838cd41978eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:43.760232 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d972e1-166b-4105-8ed3-838cd41978eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f7d972e1-166b-4105-8ed3-838cd41978eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:58:43.764918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:43.766174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:43.767127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 575/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:58:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:43.771992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1849e-4c8a-4f91-8b95-69e05c23e2b9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:43.868496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc1849e-4c8a-4f91-8b95-69e05c23e2b9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:43.902223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbdef2-dc3d-448e-931f-742f9f610255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:43.902812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbdef2-dc3d-448e-931f-742f9f610255 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:43.903087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbdef2-dc3d-448e-931f-742f9f610255 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:43.903380 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fbdef2-dc3d-448e-931f-742f9f610255 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:43.904035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbdef2-dc3d-448e-931f-742f9f610255 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:43.904333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbdef2-dc3d-448e-931f-742f9f610255 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:43.905714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fbdef2-dc3d-448e-931f-742f9f610255 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:43.914408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 582/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:58:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:43.920312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d972e1-166b-4105-8ed3-838cd41978eb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:43.995940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:44.021391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d972e1-166b-4105-8ed3-838cd41978eb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:44.054423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65e646-673b-4fb5-b7b0-ac0bab7a2eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:44.055050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65e646-673b-4fb5-b7b0-ac0bab7a2eda None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:44.055329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65e646-673b-4fb5-b7b0-ac0bab7a2eda None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:44.055601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d65e646-673b-4fb5-b7b0-ac0bab7a2eda None admin] PATH_INFO: `/v3/users/dfe1dbba21fe48ac90a8966e49a46b0f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:44.056726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65e646-673b-4fb5-b7b0-ac0bab7a2eda None admin] RBAC: Authorizing `identity:get_user(user_id=dfe1dbba21fe48ac90a8966e49a46b0f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:44.058190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d65e646-673b-4fb5-b7b0-ac0bab7a2eda None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:44.059764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 576/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:43 2022] GET /identity/v3/users/dfe1dbba21fe48ac90a8966e49a46b0f => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:44.065617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbdef2-dc3d-448e-931f-742f9f610255 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:44.084424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:44.163512 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fbdef2-dc3d-448e-931f-742f9f610255 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:44.195573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16780361-4ddb-4cec-917d-d6b00576279a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:44.196159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16780361-4ddb-4cec-917d-d6b00576279a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:44.196433 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16780361-4ddb-4cec-917d-d6b00576279a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:44.196699 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16780361-4ddb-4cec-917d-d6b00576279a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:44.260713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16780361-4ddb-4cec-917d-d6b00576279a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:44.262171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16780361-4ddb-4cec-917d-d6b00576279a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:44.278505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:44.357052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 583/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:58:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:58:44.364447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65e646-673b-4fb5-b7b0-ac0bab7a2eda None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:44.460261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d65e646-673b-4fb5-b7b0-ac0bab7a2eda None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:44.493191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba01cb-51d5-4181-bd95-49534e955602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:44.493796 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba01cb-51d5-4181-bd95-49534e955602 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:44.494067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba01cb-51d5-4181-bd95-49534e955602 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:44.494331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba01cb-51d5-4181-bd95-49534e955602 None admin] PATH_INFO: `/v3/projects/3bb2b67de0324cda962eeb67a6416e9c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:44.495414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba01cb-51d5-4181-bd95-49534e955602 None admin] RBAC: Authorizing `identity:get_project(project_id=3bb2b67de0324cda962eeb67a6416e9c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:44.496816 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba01cb-51d5-4181-bd95-49534e955602 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:44.498429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 577/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:58:44 2022] GET /identity/v3/projects/3bb2b67de0324cda962eeb67a6416e9c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:44.504592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16780361-4ddb-4cec-917d-d6b00576279a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:44.600731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16780361-4ddb-4cec-917d-d6b00576279a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:44.616633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:44.634222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c703a7ab-10da-4840-a56f-681541216d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:44.634808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703a7ab-10da-4840-a56f-681541216d32 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:44.635084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703a7ab-10da-4840-a56f-681541216d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:44.635354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c703a7ab-10da-4840-a56f-681541216d32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:44.636060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703a7ab-10da-4840-a56f-681541216d32 None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=3bb2b67de0324cda962eeb67a6416e9c, user.id=dfe1dbba21fe48ac90a8966e49a46b0f) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:44.636364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703a7ab-10da-4840-a56f-681541216d32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:44.637805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c703a7ab-10da-4840-a56f-681541216d32 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:44.643391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 584/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:58:44 2022] GET /identity/v3/role_assignments?user.id=dfe1dbba21fe48ac90a8966e49a46b0f&scope.project.id=3bb2b67de0324cda962eeb67a6416e9c&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:46.802367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ef38a-afad-42aa-a3b3-ae65ed11c0ea None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:46.802917 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ef38a-afad-42aa-a3b3-ae65ed11c0ea None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:46.802917 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ef38a-afad-42aa-a3b3-ae65ed11c0ea None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:46.804259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 578/1162] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:46.809362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cee3f-5068-4060-afd5-f35032794751 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:46.809800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cee3f-5068-4060-afd5-f35032794751 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:46.809901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1cee3f-5068-4060-afd5-f35032794751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:46.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4b1cee3f-5068-4060-afd5-f35032794751 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:46.828825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4b1cee3f-5068-4060-afd5-f35032794751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:46.894292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 585/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:47.166202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ef38a-afad-42aa-a3b3-ae65ed11c0ea None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:47.179303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:47.267021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ef38a-afad-42aa-a3b3-ae65ed11c0ea None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:47.299354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f023f-b341-4387-8822-30e7766ad899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:47.300522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f023f-b341-4387-8822-30e7766ad899 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:47.300794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f023f-b341-4387-8822-30e7766ad899 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:47.301077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f023f-b341-4387-8822-30e7766ad899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:47.364836 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f023f-b341-4387-8822-30e7766ad899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:47.366222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f023f-b341-4387-8822-30e7766ad899 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:47.376537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:47.461362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 579/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:58:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:58:47.467533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cee3f-5068-4060-afd5-f35032794751 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:47.561732 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1cee3f-5068-4060-afd5-f35032794751 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:47.571810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:47.595320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e25fa-fdc8-4d98-85d5-7e3916c5614c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:47.595926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e25fa-fdc8-4d98-85d5-7e3916c5614c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:47.596198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e25fa-fdc8-4d98-85d5-7e3916c5614c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:47.596465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e25fa-fdc8-4d98-85d5-7e3916c5614c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:47.603512 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e25fa-fdc8-4d98-85d5-7e3916c5614c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:47.604922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e25fa-fdc8-4d98-85d5-7e3916c5614c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-672e25fa-fdc8-4d98-85d5-7e3916c5614c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:47.616589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:47.617947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:58:47.619105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 586/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 22 21:58:47 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:47.623482 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f023f-b341-4387-8822-30e7766ad899 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:47.718236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f023f-b341-4387-8822-30e7766ad899 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:47.728988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:47.751832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834ffc5-e636-4735-860a-a5561450a51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:47.752421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834ffc5-e636-4735-860a-a5561450a51b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:47.752699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834ffc5-e636-4735-860a-a5561450a51b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:47.752994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834ffc5-e636-4735-860a-a5561450a51b None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:47.753739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834ffc5-e636-4735-860a-a5561450a51b None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:47.754041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834ffc5-e636-4735-860a-a5561450a51b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:47.755397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834ffc5-e636-4735-860a-a5561450a51b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:47.762235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 580/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Thu Sep 22 21:58:47 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:49.911817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33538df6-ce2b-404e-b820-2ea983ff53ec None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:49.912356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33538df6-ce2b-404e-b820-2ea983ff53ec None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:49.912356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33538df6-ce2b-404e-b820-2ea983ff53ec None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:49.913796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 587/1167] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:49.919163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd46be-4183-4794-b8fe-2a5c39316b9c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:49.919542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd46be-4183-4794-b8fe-2a5c39316b9c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:49.919700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffd46be-4183-4794-b8fe-2a5c39316b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:49.937369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2ffd46be-4183-4794-b8fe-2a5c39316b9c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:49.938539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2ffd46be-4183-4794-b8fe-2a5c39316b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:50.002267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 581/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 21:58:50.277810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33538df6-ce2b-404e-b820-2ea983ff53ec None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:50.374363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33538df6-ce2b-404e-b820-2ea983ff53ec None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:50.407420 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2ac7f-77e6-416c-8c2b-a28361b5bcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:50.408011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2ac7f-77e6-416c-8c2b-a28361b5bcc7 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:50.408286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2ac7f-77e6-416c-8c2b-a28361b5bcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:50.408553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2ac7f-77e6-416c-8c2b-a28361b5bcc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:50.409257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2ac7f-77e6-416c-8c2b-a28361b5bcc7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:50.410598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2ac7f-77e6-416c-8c2b-a28361b5bcc7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:50.426188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 588/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Thu Sep 22 21:58:50 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:58:52.627133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca246d-38ba-4b2c-8cd7-657f42d4a857 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:52.627700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca246d-38ba-4b2c-8cd7-657f42d4a857 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:52.627820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca246d-38ba-4b2c-8cd7-657f42d4a857 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:52.629401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 582/1170] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:52.634800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ab1d2-3ec6-4e7f-ba90-d7cc6e86672f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:52.635206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ab1d2-3ec6-4e7f-ba90-d7cc6e86672f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:52.635346 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ab1d2-3ec6-4e7f-ba90-d7cc6e86672f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:52.653403 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-081ab1d2-3ec6-4e7f-ba90-d7cc6e86672f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:52.654587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-081ab1d2-3ec6-4e7f-ba90-d7cc6e86672f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:52.678483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:52.720197 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 589/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:52.994411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca246d-38ba-4b2c-8cd7-657f42d4a857 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:53.093022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca246d-38ba-4b2c-8cd7-657f42d4a857 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:53.095268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:53.126330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8500a9-5695-4612-8920-88b51dd9910b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:53.126943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8500a9-5695-4612-8920-88b51dd9910b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:53.127215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8500a9-5695-4612-8920-88b51dd9910b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:53.127511 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8500a9-5695-4612-8920-88b51dd9910b None admin] PATH_INFO: `/v3/domains/491a1342176c4c42acd7e30f3bbfcf39` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:53.137490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8500a9-5695-4612-8920-88b51dd9910b None admin] RBAC: Authorizing `identity:get_domain(domain_id=491a1342176c4c42acd7e30f3bbfcf39)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:53.138913 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8500a9-5695-4612-8920-88b51dd9910b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:53.140484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 583/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Sep 22 21:58:52 2022] GET /identity/v3/domains/491a1342176c4c42acd7e30f3bbfcf39 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:53.147615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ab1d2-3ec6-4e7f-ba90-d7cc6e86672f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:53.248027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ab1d2-3ec6-4e7f-ba90-d7cc6e86672f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:53.281171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff41232a-84c2-4b13-815a-2a061b9188d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:53.281762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41232a-84c2-4b13-815a-2a061b9188d3 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:53.282037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41232a-84c2-4b13-815a-2a061b9188d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:53.282306 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff41232a-84c2-4b13-815a-2a061b9188d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:53.282947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41232a-84c2-4b13-815a-2a061b9188d3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:53.284391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff41232a-84c2-4b13-815a-2a061b9188d3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:53.309643 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 590/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 21:58:53 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:58:55.491617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbbc8d-7521-4cfb-9370-476dd02ff07f None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:55.492132 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbbc8d-7521-4cfb-9370-476dd02ff07f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:55.492132 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdbbc8d-7521-4cfb-9370-476dd02ff07f None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:55.493449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 584/1174] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:55.498727 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2239a-0e0b-498f-8bf5-fffdfe11f2be None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:55.499134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2239a-0e0b-498f-8bf5-fffdfe11f2be None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:55.499277 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2239a-0e0b-498f-8bf5-fffdfe11f2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:55.517480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ffe2239a-0e0b-498f-8bf5-fffdfe11f2be None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:55.519261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ffe2239a-0e0b-498f-8bf5-fffdfe11f2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:55.526329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:55.584670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 591/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:55.855149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbbc8d-7521-4cfb-9370-476dd02ff07f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:55.965457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdbbc8d-7521-4cfb-9370-476dd02ff07f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:55.998391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef67fbfc-4fb5-4068-855c-e5fd7966206f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:55.999007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67fbfc-4fb5-4068-855c-e5fd7966206f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:55.999285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67fbfc-4fb5-4068-855c-e5fd7966206f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:55.999557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef67fbfc-4fb5-4068-855c-e5fd7966206f None admin] PATH_INFO: `/v3/domains/491a1342176c4c42acd7e30f3bbfcf39` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:56.000664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67fbfc-4fb5-4068-855c-e5fd7966206f None admin] RBAC: Authorizing `identity:get_domain(domain_id=491a1342176c4c42acd7e30f3bbfcf39)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:56.002119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef67fbfc-4fb5-4068-855c-e5fd7966206f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:56.004364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 585/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Thu Sep 22 21:58:55 2022] GET /identity/v3/domains/491a1342176c4c42acd7e30f3bbfcf39 => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:56.010283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2239a-0e0b-498f-8bf5-fffdfe11f2be None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:56.104696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2239a-0e0b-498f-8bf5-fffdfe11f2be None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:56.138476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbef7b6-de2b-44da-b6ac-9eff16bc4ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:56.139018 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbef7b6-de2b-44da-b6ac-9eff16bc4ef3 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:56.139294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbef7b6-de2b-44da-b6ac-9eff16bc4ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:56.139566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbef7b6-de2b-44da-b6ac-9eff16bc4ef3 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:56.140206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbef7b6-de2b-44da-b6ac-9eff16bc4ef3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:56.141655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbef7b6-de2b-44da-b6ac-9eff16bc4ef3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:56.158060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:56.164591 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 592/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 21:58:56 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 21:58:58.347462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e2c48-76de-40a4-8124-68c629586f4c None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:58.347964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e2c48-76de-40a4-8124-68c629586f4c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:58.347964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e2c48-76de-40a4-8124-68c629586f4c None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:58.349272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 586/1178] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:58:58.354527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f1db-b2e7-45c2-8097-a157e6e3c894 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:58.354939 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f1db-b2e7-45c2-8097-a157e6e3c894 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:58.355077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0f1db-b2e7-45c2-8097-a157e6e3c894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:58.373389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bed0f1db-b2e7-45c2-8097-a157e6e3c894 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:58:58.374570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bed0f1db-b2e7-45c2-8097-a157e6e3c894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:58:58.439360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 593/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:58:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:58:58.712320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e2c48-76de-40a4-8124-68c629586f4c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:58.743923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:58.811259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e2c48-76de-40a4-8124-68c629586f4c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:58.843686 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f59e9-910a-405f-aac6-81be89b90761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:58.844294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f59e9-910a-405f-aac6-81be89b90761 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:58.844569 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f59e9-910a-405f-aac6-81be89b90761 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:58.844834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f59e9-910a-405f-aac6-81be89b90761 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:58.855026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f59e9-910a-405f-aac6-81be89b90761 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-ae0f59e9-910a-405f-aac6-81be89b90761 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:58:58.859711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:58:58.860837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:58:58.862142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 587/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:58:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:58:58.866799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f1db-b2e7-45c2-8097-a157e6e3c894 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:58.964906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0f1db-b2e7-45c2-8097-a157e6e3c894 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:58.990178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:58.997797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36b752-d4d5-436c-a270-ce05ef1f74d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:58.998390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b752-d4d5-436c-a270-ce05ef1f74d6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:58.998662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b752-d4d5-436c-a270-ce05ef1f74d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:58.998928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a36b752-d4d5-436c-a270-ce05ef1f74d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:58.999570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36b752-d4d5-436c-a270-ce05ef1f74d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:58.999865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36b752-d4d5-436c-a270-ce05ef1f74d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:59.001231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a36b752-d4d5-436c-a270-ce05ef1f74d6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:59.008912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 594/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:58:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 21:58:59.014750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f59e9-910a-405f-aac6-81be89b90761 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:59.103835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:59.112762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f59e9-910a-405f-aac6-81be89b90761 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:59.145199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c1d07-dd3c-4f10-87d2-a565e859186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:59.145807 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c1d07-dd3c-4f10-87d2-a565e859186f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:59.146079 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c1d07-dd3c-4f10-87d2-a565e859186f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:59.146345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c1d07-dd3c-4f10-87d2-a565e859186f None admin] PATH_INFO: `/v3/users/bc2198c6c6e64f39a4ea7c3d9cd58f65` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:59.159320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c1d07-dd3c-4f10-87d2-a565e859186f None admin] RBAC: Authorizing `identity:get_user(user_id=bc2198c6c6e64f39a4ea7c3d9cd58f65)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:59.160727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c1d07-dd3c-4f10-87d2-a565e859186f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:59.162327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 588/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:58:59 2022] GET /identity/v3/users/bc2198c6c6e64f39a4ea7c3d9cd58f65 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:59.168223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36b752-d4d5-436c-a270-ce05ef1f74d6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:59.263187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a36b752-d4d5-436c-a270-ce05ef1f74d6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:59.294951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0dd91-b150-4fb0-9960-8d897de5917c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:59.295538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0dd91-b150-4fb0-9960-8d897de5917c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:59.295812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0dd91-b150-4fb0-9960-8d897de5917c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:59.296078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c0dd91-b150-4fb0-9960-8d897de5917c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:59.350152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:59.360556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0dd91-b150-4fb0-9960-8d897de5917c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:59.362244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c0dd91-b150-4fb0-9960-8d897de5917c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:59.456783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 595/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:58:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 21:58:59.462995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c1d07-dd3c-4f10-87d2-a565e859186f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:59.560883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c1d07-dd3c-4f10-87d2-a565e859186f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:59.594676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1472a-3679-487a-9d77-5419d4f68bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:59.595287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1472a-3679-487a-9d77-5419d4f68bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:59.595565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1472a-3679-487a-9d77-5419d4f68bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:59.595885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1472a-3679-487a-9d77-5419d4f68bc3 None admin] PATH_INFO: `/v3/projects/e23f10922d184f6c95dd22aa5b5dd090` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:59.597477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1472a-3679-487a-9d77-5419d4f68bc3 None admin] RBAC: Authorizing `identity:get_project(project_id=e23f10922d184f6c95dd22aa5b5dd090)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:59.598903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1472a-3679-487a-9d77-5419d4f68bc3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:59.600504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 589/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:58:59 2022] GET /identity/v3/projects/e23f10922d184f6c95dd22aa5b5dd090 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:58:59.606564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0dd91-b150-4fb0-9960-8d897de5917c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:58:59.692679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:58:59.701849 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c0dd91-b150-4fb0-9960-8d897de5917c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:58:59.734353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c67c0-e439-47bf-9e59-50dda6cf10a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:58:59.734936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c67c0-e439-47bf-9e59-50dda6cf10a4 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:58:59.735209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c67c0-e439-47bf-9e59-50dda6cf10a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:58:59.735478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c67c0-e439-47bf-9e59-50dda6cf10a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:58:59.736188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c67c0-e439-47bf-9e59-50dda6cf10a4 None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=e23f10922d184f6c95dd22aa5b5dd090, user.id=bc2198c6c6e64f39a4ea7c3d9cd58f65) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:58:59.737068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c67c0-e439-47bf-9e59-50dda6cf10a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:58:59.738467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3c67c0-e439-47bf-9e59-50dda6cf10a4 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:58:59.743679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 596/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:58:59 2022] GET /identity/v3/role_assignments?user.id=bc2198c6c6e64f39a4ea7c3d9cd58f65&scope.project.id=e23f10922d184f6c95dd22aa5b5dd090&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:59:01.901053 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239414cd-cbfa-49dd-83a9-e362c16dd23b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:01.901642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239414cd-cbfa-49dd-83a9-e362c16dd23b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:01.901736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239414cd-cbfa-49dd-83a9-e362c16dd23b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:01.903077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 590/1186] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:59:01.909044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7860f5-017c-4add-8f9e-294fe48a8144 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:01.909560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7860f5-017c-4add-8f9e-294fe48a8144 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:01.909843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7860f5-017c-4add-8f9e-294fe48a8144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:01.927895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8d7860f5-017c-4add-8f9e-294fe48a8144 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:59:01.929084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8d7860f5-017c-4add-8f9e-294fe48a8144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:59:01.994949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 597/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:59:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:59:02.270492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239414cd-cbfa-49dd-83a9-e362c16dd23b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:59:02.288722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:59:02.367269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239414cd-cbfa-49dd-83a9-e362c16dd23b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:59:02.400008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8428e2-1f91-4610-a7c0-aca92dfaeccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:59:02.400612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8428e2-1f91-4610-a7c0-aca92dfaeccd None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:02.400885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8428e2-1f91-4610-a7c0-aca92dfaeccd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:02.401174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8428e2-1f91-4610-a7c0-aca92dfaeccd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:02.410852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8428e2-1f91-4610-a7c0-aca92dfaeccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-1e8428e2-1f91-4610-a7c0-aca92dfaeccd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:59:02.415497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:59:02.416609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 21:59:02.417683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 591/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:59:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:59:02.422554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7860f5-017c-4add-8f9e-294fe48a8144 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:59:02.502593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:59:02.518419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7860f5-017c-4add-8f9e-294fe48a8144 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:59:02.550828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54f029-12b4-4676-8aa7-87f67b21c528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:59:02.551356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54f029-12b4-4676-8aa7-87f67b21c528 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:02.551627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54f029-12b4-4676-8aa7-87f67b21c528 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:02.551892 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54f029-12b4-4676-8aa7-87f67b21c528 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:02.552532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54f029-12b4-4676-8aa7-87f67b21c528 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:59:02.552826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54f029-12b4-4676-8aa7-87f67b21c528 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:59:02.554197 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54f029-12b4-4676-8aa7-87f67b21c528 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:59:02.561791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 598/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:59:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:59:02.568003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8428e2-1f91-4610-a7c0-aca92dfaeccd None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:59:02.643801 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:59:02.664414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8428e2-1f91-4610-a7c0-aca92dfaeccd None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:59:02.697671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff5417-9296-4310-a2d6-7a7b7fbb83ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:59:02.698281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff5417-9296-4310-a2d6-7a7b7fbb83ce None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:02.698559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff5417-9296-4310-a2d6-7a7b7fbb83ce None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:02.698828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff5417-9296-4310-a2d6-7a7b7fbb83ce None admin] PATH_INFO: `/v3/users/bc2198c6c6e64f39a4ea7c3d9cd58f65` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:02.699928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff5417-9296-4310-a2d6-7a7b7fbb83ce None admin] RBAC: Authorizing `identity:get_user(user_id=bc2198c6c6e64f39a4ea7c3d9cd58f65)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:59:02.701439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff5417-9296-4310-a2d6-7a7b7fbb83ce None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:59:02.702992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 592/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:59:02 2022] GET /identity/v3/users/bc2198c6c6e64f39a4ea7c3d9cd58f65 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:59:02.708886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54f029-12b4-4676-8aa7-87f67b21c528 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:59:02.806419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54f029-12b4-4676-8aa7-87f67b21c528 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:59:02.838579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb3c0ad-ec8a-4e4f-a578-d8a023796dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:59:02.839191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3c0ad-ec8a-4e4f-a578-d8a023796dcf None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:02.839468 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3c0ad-ec8a-4e4f-a578-d8a023796dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:02.839736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb3c0ad-ec8a-4e4f-a578-d8a023796dcf None admin] PATH_INFO: `/v3/projects/e23f10922d184f6c95dd22aa5b5dd090` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:02.840824 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3c0ad-ec8a-4e4f-a578-d8a023796dcf None admin] RBAC: Authorizing `identity:get_project(project_id=e23f10922d184f6c95dd22aa5b5dd090)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:59:02.842270 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb3c0ad-ec8a-4e4f-a578-d8a023796dcf None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:59:02.843861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 599/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:59:02 2022] GET /identity/v3/projects/e23f10922d184f6c95dd22aa5b5dd090 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:59:02.850027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff5417-9296-4310-a2d6-7a7b7fbb83ce None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:59:02.947210 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff5417-9296-4310-a2d6-7a7b7fbb83ce None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:59:02.979519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe226400-ef7d-4fd2-8dcf-264265d462bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:59:02.980155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe226400-ef7d-4fd2-8dcf-264265d462bd None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:02.980431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe226400-ef7d-4fd2-8dcf-264265d462bd None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:02.980704 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe226400-ef7d-4fd2-8dcf-264265d462bd None admin] PATH_INFO: `/v3/projects/e23f10922d184f6c95dd22aa5b5dd090/users/bc2198c6c6e64f39a4ea7c3d9cd58f65/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:02.982605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe226400-ef7d-4fd2-8dcf-264265d462bd None admin] RBAC: Authorizing `identity:create_grant(project_id=e23f10922d184f6c95dd22aa5b5dd090, user_id=bc2198c6c6e64f39a4ea7c3d9cd58f65, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:59:02.984017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe226400-ef7d-4fd2-8dcf-264265d462bd None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:59:02.996777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 593/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 21:59:02 2022] PUT /identity/v3/projects/e23f10922d184f6c95dd22aa5b5dd090/users/bc2198c6c6e64f39a4ea7c3d9cd58f65/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 21:59:05.117847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847b983-017e-4cef-8bd4-3f532216ec8b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:05.118328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847b983-017e-4cef-8bd4-3f532216ec8b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:05.118418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3847b983-017e-4cef-8bd4-3f532216ec8b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:05.119738 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 600/1193] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 21:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 21:59:05.124822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34cb2-8abc-4753-8a69-8afbaa086bf1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:05.125279 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34cb2-8abc-4753-8a69-8afbaa086bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:05.125394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae34cb2-8abc-4753-8a69-8afbaa086bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:05.138561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:59:05.144810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5ae34cb2-8abc-4753-8a69-8afbaa086bf1 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 21:59:05.146194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5ae34cb2-8abc-4753-8a69-8afbaa086bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 21:59:05.210949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 594/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 21:59:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 21:59:05.478859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847b983-017e-4cef-8bd4-3f532216ec8b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:59:05.494826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:59:05.578488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3847b983-017e-4cef-8bd4-3f532216ec8b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:59:05.610252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8635e8-d8f0-43b4-8cf6-2626b99ddc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:59:05.610864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8635e8-d8f0-43b4-8cf6-2626b99ddc31 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:05.611689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8635e8-d8f0-43b4-8cf6-2626b99ddc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:05.612033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8635e8-d8f0-43b4-8cf6-2626b99ddc31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:05.622127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8635e8-d8f0-43b4-8cf6-2626b99ddc31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-0d8635e8-d8f0-43b4-8cf6-2626b99ddc31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 21:59:05.627421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 21:59:05.628602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 21:59:05.629844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 601/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 21:59:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 21:59:05.634425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae34cb2-8abc-4753-8a69-8afbaa086bf1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:59:05.728694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae34cb2-8abc-4753-8a69-8afbaa086bf1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:59:05.761550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8691f-55a3-4aab-a489-c82797cc893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:59:05.762145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8691f-55a3-4aab-a489-c82797cc893f None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:05.762424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8691f-55a3-4aab-a489-c82797cc893f None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:05.762696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8691f-55a3-4aab-a489-c82797cc893f None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:05.763345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8691f-55a3-4aab-a489-c82797cc893f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:59:05.763654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8691f-55a3-4aab-a489-c82797cc893f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:59:05.765011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8691f-55a3-4aab-a489-c82797cc893f None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:59:05.767118 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:59:05.773553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 595/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 21:59:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:59:05.779649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8635e8-d8f0-43b4-8cf6-2626b99ddc31 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:59:05.851457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:59:05.876030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8635e8-d8f0-43b4-8cf6-2626b99ddc31 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:59:05.908769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1411ddb-035f-49e1-9f1a-30a2c1e996b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:59:05.909397 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1411ddb-035f-49e1-9f1a-30a2c1e996b6 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:05.909674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1411ddb-035f-49e1-9f1a-30a2c1e996b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:05.909946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1411ddb-035f-49e1-9f1a-30a2c1e996b6 None admin] PATH_INFO: `/v3/users/bc2198c6c6e64f39a4ea7c3d9cd58f65` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:05.911042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1411ddb-035f-49e1-9f1a-30a2c1e996b6 None admin] RBAC: Authorizing `identity:get_user(user_id=bc2198c6c6e64f39a4ea7c3d9cd58f65)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:59:05.912453 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1411ddb-035f-49e1-9f1a-30a2c1e996b6 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:59:05.914022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 602/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Thu Sep 22 21:59:05 2022] GET /identity/v3/users/bc2198c6c6e64f39a4ea7c3d9cd58f65 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:59:05.919926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8691f-55a3-4aab-a489-c82797cc893f None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:59:06.026555 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8691f-55a3-4aab-a489-c82797cc893f None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:59:06.059594 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf692a8-63ca-4b62-8c25-fc51473ce3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:59:06.060193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf692a8-63ca-4b62-8c25-fc51473ce3bf None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:06.060469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf692a8-63ca-4b62-8c25-fc51473ce3bf None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:06.060741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf692a8-63ca-4b62-8c25-fc51473ce3bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:06.123031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:59:06.127274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf692a8-63ca-4b62-8c25-fc51473ce3bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:59:06.128706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf692a8-63ca-4b62-8c25-fc51473ce3bf None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:59:06.223493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 596/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 21:59:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 21:59:06.229441 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1411ddb-035f-49e1-9f1a-30a2c1e996b6 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:59:06.326991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1411ddb-035f-49e1-9f1a-30a2c1e996b6 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:59:06.360246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1209a-1ab0-4ebb-8abb-59fec419e11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:59:06.360859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1209a-1ab0-4ebb-8abb-59fec419e11e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:06.361250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1209a-1ab0-4ebb-8abb-59fec419e11e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:06.361530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1209a-1ab0-4ebb-8abb-59fec419e11e None admin] PATH_INFO: `/v3/projects/e23f10922d184f6c95dd22aa5b5dd090` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:06.362691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1209a-1ab0-4ebb-8abb-59fec419e11e None admin] RBAC: Authorizing `identity:get_project(project_id=e23f10922d184f6c95dd22aa5b5dd090)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:59:06.364100 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1209a-1ab0-4ebb-8abb-59fec419e11e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:59:06.365747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 603/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Thu Sep 22 21:59:06 2022] GET /identity/v3/projects/e23f10922d184f6c95dd22aa5b5dd090 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:59:06.371884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf692a8-63ca-4b62-8c25-fc51473ce3bf None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 21:59:06.463174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:59:06.467061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf692a8-63ca-4b62-8c25-fc51473ce3bf None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 21:59:06.499771 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07458e6-ed66-4949-aedb-d4e86618afba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 21:59:06.500353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07458e6-ed66-4949-aedb-d4e86618afba None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 21:59:06.500628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07458e6-ed66-4949-aedb-d4e86618afba None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 21:59:06.500897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07458e6-ed66-4949-aedb-d4e86618afba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 21:59:06.501668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07458e6-ed66-4949-aedb-d4e86618afba None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=e23f10922d184f6c95dd22aa5b5dd090, user.id=bc2198c6c6e64f39a4ea7c3d9cd58f65) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 21:59:06.501966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07458e6-ed66-4949-aedb-d4e86618afba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 21:59:06.503376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07458e6-ed66-4949-aedb-d4e86618afba None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 21:59:06.508868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 597/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 21:59:06 2022] GET /identity/v3/role_assignments?user.id=bc2198c6c6e64f39a4ea7c3d9cd58f65&scope.project.id=e23f10922d184f6c95dd22aa5b5dd090&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 21:59:16.356282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 21:59:16.506373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=19 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:02.664085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cacdd2-4bb4-4ad3-a2c9-087db034796f None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:02.664680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cacdd2-4bb4-4ad3-a2c9-087db034796f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:02.664680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cacdd2-4bb4-4ad3-a2c9-087db034796f None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:02.666120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 604/1201] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:00:02.675487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a19581-fc51-4867-8f85-87ab6b693b6e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:02.675487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a19581-fc51-4867-8f85-87ab6b693b6e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:02.675487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a19581-fc51-4867-8f85-87ab6b693b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:02.695065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c6a19581-fc51-4867-8f85-87ab6b693b6e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:00:02.696278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c6a19581-fc51-4867-8f85-87ab6b693b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:00:02.790506 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 598/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:00:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 22:00:03.079014 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cacdd2-4bb4-4ad3-a2c9-087db034796f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:03.226537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cacdd2-4bb4-4ad3-a2c9-087db034796f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:03.277510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f100e-3027-4009-a73f-bca839239c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:03.278246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f100e-3027-4009-a73f-bca839239c10 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:03.278587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f100e-3027-4009-a73f-bca839239c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:03.278976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f100e-3027-4009-a73f-bca839239c10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:03.290682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f100e-3027-4009-a73f-bca839239c10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:03.292150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f100e-3027-4009-a73f-bca839239c10 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-300f100e-3027-4009-a73f-bca839239c10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:03.316887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:03.318063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:03.319041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:03.319041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:03.319041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:03.326556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 605/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:03.328128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a19581-fc51-4867-8f85-87ab6b693b6e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:03.465045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a19581-fc51-4867-8f85-87ab6b693b6e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:03.504557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:03.512283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820eabe-56e3-438d-88fd-822c511f81a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:03.513613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820eabe-56e3-438d-88fd-822c511f81a7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:03.513891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820eabe-56e3-438d-88fd-822c511f81a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:03.514227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820eabe-56e3-438d-88fd-822c511f81a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:03.514992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820eabe-56e3-438d-88fd-822c511f81a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:03.515369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820eabe-56e3-438d-88fd-822c511f81a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:03.516907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820eabe-56e3-438d-88fd-822c511f81a7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:03.535690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 599/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:03.541860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f100e-3027-4009-a73f-bca839239c10 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:03.608669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:03.670671 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f100e-3027-4009-a73f-bca839239c10 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:03.715947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09029699-bed6-4b2f-8871-c9e235078ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:03.717487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029699-bed6-4b2f-8871-c9e235078ae1 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:03.718072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029699-bed6-4b2f-8871-c9e235078ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:03.718496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09029699-bed6-4b2f-8871-c9e235078ae1 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:03.719351 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09029699-bed6-4b2f-8871-c9e235078ae1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:03.721291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09029699-bed6-4b2f-8871-c9e235078ae1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:03.756608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 606/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Thu Sep 22 22:00:03 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:00:06.138271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee652111-0c05-44c2-8d06-49f9fdbdf050 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:06.138832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee652111-0c05-44c2-8d06-49f9fdbdf050 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:06.138961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee652111-0c05-44c2-8d06-49f9fdbdf050 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:06.140324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 600/1206] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 22:00:06.146069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e839-d441-4ad9-a5bd-4e6cd2c10040 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:06.146602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e839-d441-4ad9-a5bd-4e6cd2c10040 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:06.146602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e839-d441-4ad9-a5bd-4e6cd2c10040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:06.165817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ad08e839-d441-4ad9-a5bd-4e6cd2c10040 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:00:06.167016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ad08e839-d441-4ad9-a5bd-4e6cd2c10040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:00:06.245538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 607/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:00:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 22:00:06.517559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee652111-0c05-44c2-8d06-49f9fdbdf050 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:06.624492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee652111-0c05-44c2-8d06-49f9fdbdf050 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:06.665824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4fe29-dd07-4573-916c-7dc7bfd86e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:06.666433 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4fe29-dd07-4573-916c-7dc7bfd86e11 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:06.666731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4fe29-dd07-4573-916c-7dc7bfd86e11 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:06.666967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4fe29-dd07-4573-916c-7dc7bfd86e11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:06.677505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4fe29-dd07-4573-916c-7dc7bfd86e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-9fb4fe29-dd07-4573-916c-7dc7bfd86e11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 22:00:06.682597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:06.683622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:06.684513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 601/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 22:00:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:06.690094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08e839-d441-4ad9-a5bd-4e6cd2c10040 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:06.740357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:06.809440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08e839-d441-4ad9-a5bd-4e6cd2c10040 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:06.858730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2989e1b-4fd5-43ea-ab2d-ad9fc29e8af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:06.859718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2989e1b-4fd5-43ea-ab2d-ad9fc29e8af9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:06.860230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2989e1b-4fd5-43ea-ab2d-ad9fc29e8af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:06.860621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2989e1b-4fd5-43ea-ab2d-ad9fc29e8af9 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:06.861540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2989e1b-4fd5-43ea-ab2d-ad9fc29e8af9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:06.861844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2989e1b-4fd5-43ea-ab2d-ad9fc29e8af9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:06.863581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2989e1b-4fd5-43ea-ab2d-ad9fc29e8af9 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:06.873343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 608/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 22:00:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:00:06.879665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4fe29-dd07-4573-916c-7dc7bfd86e11 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:06.936635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:06.994602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4fe29-dd07-4573-916c-7dc7bfd86e11 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:07.038185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb95a5-3039-44a9-bb66-724557a70872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:07.038815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb95a5-3039-44a9-bb66-724557a70872 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:07.039094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb95a5-3039-44a9-bb66-724557a70872 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:07.039366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb95a5-3039-44a9-bb66-724557a70872 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:07.055463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb95a5-3039-44a9-bb66-724557a70872 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:07.057400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb95a5-3039-44a9-bb66-724557a70872 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-6ecb95a5-3039-44a9-bb66-724557a70872 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:07.080950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:07.082164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:07.083395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:07.083395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:07.083395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:07.083395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 602/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 22:00:07.087734 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2989e1b-4fd5-43ea-ab2d-ad9fc29e8af9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:07.214374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:07.218198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2989e1b-4fd5-43ea-ab2d-ad9fc29e8af9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:07.263001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d1aad-1744-4563-94a3-7ed9b123d123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:07.263749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d1aad-1744-4563-94a3-7ed9b123d123 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:07.264085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d1aad-1744-4563-94a3-7ed9b123d123 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:07.264383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d1aad-1744-4563-94a3-7ed9b123d123 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:07.265047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d1aad-1744-4563-94a3-7ed9b123d123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:07.265394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d1aad-1744-4563-94a3-7ed9b123d123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:07.267036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033d1aad-1744-4563-94a3-7ed9b123d123 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:07.285342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 609/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:00:07.290924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb95a5-3039-44a9-bb66-724557a70872 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:07.411590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb95a5-3039-44a9-bb66-724557a70872 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:07.412608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:07.451076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db439112-5ae6-43b4-a18a-46e3cba8a299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:07.451708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db439112-5ae6-43b4-a18a-46e3cba8a299 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:07.451983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db439112-5ae6-43b4-a18a-46e3cba8a299 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:07.452252 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db439112-5ae6-43b4-a18a-46e3cba8a299 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:07.459939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db439112-5ae6-43b4-a18a-46e3cba8a299 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:07.461395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db439112-5ae6-43b4-a18a-46e3cba8a299 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-db439112-5ae6-43b4-a18a-46e3cba8a299 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 22:00:07.476007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:07.482846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:07.484000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:07.484000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:07.484000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 22:00:07.484000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 22:00:07.484000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 22:00:07.484000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 22:00:07.484000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 22:00:07.484000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 22:00:07.484000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:07.484000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:07.484000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 603/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:07.486799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d1aad-1744-4563-94a3-7ed9b123d123 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:07.622307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d1aad-1744-4563-94a3-7ed9b123d123 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:07.663362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7e73b-4f47-41c7-a242-1fc31cdbda71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:07.663899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7e73b-4f47-41c7-a242-1fc31cdbda71 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:07.664172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7e73b-4f47-41c7-a242-1fc31cdbda71 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:07.664435 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f7e73b-4f47-41c7-a242-1fc31cdbda71 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:07.665178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7e73b-4f47-41c7-a242-1fc31cdbda71 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:07.665476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7e73b-4f47-41c7-a242-1fc31cdbda71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:07.666874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f7e73b-4f47-41c7-a242-1fc31cdbda71 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:07.689137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 610/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 22 22:00:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:07.692387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db439112-5ae6-43b4-a18a-46e3cba8a299 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:07.803359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db439112-5ae6-43b4-a18a-46e3cba8a299 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:07.840806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b0775-d4c1-4d40-a952-de8e23447fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:07.842483 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b0775-d4c1-4d40-a952-de8e23447fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:07.842769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b0775-d4c1-4d40-a952-de8e23447fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:07.843040 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1b0775-d4c1-4d40-a952-de8e23447fd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:07.873690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:07.919255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b0775-d4c1-4d40-a952-de8e23447fd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:07.920864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1b0775-d4c1-4d40-a952-de8e23447fd2 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:08.028028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 604/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 22:00:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 22:00:08.034305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7e73b-4f47-41c7-a242-1fc31cdbda71 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:08.055353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:08.154882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f7e73b-4f47-41c7-a242-1fc31cdbda71 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:08.192333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbde5dd-df41-4dd7-84cc-7507872b2099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:08.192887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbde5dd-df41-4dd7-84cc-7507872b2099 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:08.193183 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbde5dd-df41-4dd7-84cc-7507872b2099 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:08.193459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbde5dd-df41-4dd7-84cc-7507872b2099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:08.205471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbde5dd-df41-4dd7-84cc-7507872b2099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:08.207051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbde5dd-df41-4dd7-84cc-7507872b2099 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-8cbde5dd-df41-4dd7-84cc-7507872b2099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:08.223333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:08.224807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:08.225781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:08.225781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:08.225781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:08.225781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 611/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 22:00:08.230160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b0775-d4c1-4d40-a952-de8e23447fd2 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:08.308656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:08.355021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1b0775-d4c1-4d40-a952-de8e23447fd2 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:08.394915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f26708-a5a6-4607-bb12-bc335f2118aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:08.395517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26708-a5a6-4607-bb12-bc335f2118aa None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:08.395789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26708-a5a6-4607-bb12-bc335f2118aa None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:08.396057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f26708-a5a6-4607-bb12-bc335f2118aa None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:08.396687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f26708-a5a6-4607-bb12-bc335f2118aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:08.396983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f26708-a5a6-4607-bb12-bc335f2118aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:08.398391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f26708-a5a6-4607-bb12-bc335f2118aa None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:08.412110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 605/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:08.417895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbde5dd-df41-4dd7-84cc-7507872b2099 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:08.504055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:08.539818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbde5dd-df41-4dd7-84cc-7507872b2099 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:08.596110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0994ed-b4e0-47fd-a736-b78857c933f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:08.596949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0994ed-b4e0-47fd-a736-b78857c933f1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:08.597284 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0994ed-b4e0-47fd-a736-b78857c933f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:08.597606 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0994ed-b4e0-47fd-a736-b78857c933f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:08.607651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0994ed-b4e0-47fd-a736-b78857c933f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:08.609407 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0994ed-b4e0-47fd-a736-b78857c933f1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-dc0994ed-b4e0-47fd-a736-b78857c933f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 22:00:08.625573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:08.626797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:08.628107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 22:00:08.628107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 22:00:08.628107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 22:00:08.628107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 22:00:08.628107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 22:00:08.628107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 22:00:08.628107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:08.628107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:08.628107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 612/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 22:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:08.633915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f26708-a5a6-4607-bb12-bc335f2118aa None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:08.783777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f26708-a5a6-4607-bb12-bc335f2118aa None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:08.808932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:08.820873 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4273dc-ff6b-4de1-a472-91497b648e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:08.821509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4273dc-ff6b-4de1-a472-91497b648e38 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:08.821786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4273dc-ff6b-4de1-a472-91497b648e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:08.822065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4273dc-ff6b-4de1-a472-91497b648e38 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:08.822749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4273dc-ff6b-4de1-a472-91497b648e38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:08.823049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4273dc-ff6b-4de1-a472-91497b648e38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:08.824537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4273dc-ff6b-4de1-a472-91497b648e38 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:08.838843 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 606/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 22:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:08.846046 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0994ed-b4e0-47fd-a736-b78857c933f1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:08.957601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0994ed-b4e0-47fd-a736-b78857c933f1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:08.987912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:08.994190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea945dec-16ea-4537-aaa1-1fe0d4fe7421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:08.994792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea945dec-16ea-4537-aaa1-1fe0d4fe7421 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:08.995071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea945dec-16ea-4537-aaa1-1fe0d4fe7421 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:08.995340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea945dec-16ea-4537-aaa1-1fe0d4fe7421 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:08.996054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea945dec-16ea-4537-aaa1-1fe0d4fe7421 None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=43d6ba7b68884509a4e2859ceb72ef0e) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:08.996362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea945dec-16ea-4537-aaa1-1fe0d4fe7421 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:08.998756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea945dec-16ea-4537-aaa1-1fe0d4fe7421 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:09.004045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 613/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 22:00:08 2022] GET /identity/v3/role_assignments?user.id=43d6ba7b68884509a4e2859ceb72ef0e&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:11.170997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2756fd8-915b-42ed-aa0e-6cd7ffdae99c None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:11.171550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2756fd8-915b-42ed-aa0e-6cd7ffdae99c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:11.171550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2756fd8-915b-42ed-aa0e-6cd7ffdae99c None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:11.172860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 607/1220] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:00:11.178074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68efe5-c2c2-4cd5-a810-0c46d46b8122 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:11.178523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68efe5-c2c2-4cd5-a810-0c46d46b8122 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:11.178634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a68efe5-c2c2-4cd5-a810-0c46d46b8122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:11.196849 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6a68efe5-c2c2-4cd5-a810-0c46d46b8122 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:00:11.198073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6a68efe5-c2c2-4cd5-a810-0c46d46b8122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:00:11.268368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 614/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:00:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 22:00:11.539963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2756fd8-915b-42ed-aa0e-6cd7ffdae99c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:11.646930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2756fd8-915b-42ed-aa0e-6cd7ffdae99c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:11.681852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6181038-22da-4f97-84a3-7294fa44cf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:11.683099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6181038-22da-4f97-84a3-7294fa44cf04 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:11.683391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6181038-22da-4f97-84a3-7294fa44cf04 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:11.683681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6181038-22da-4f97-84a3-7294fa44cf04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:11.694601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6181038-22da-4f97-84a3-7294fa44cf04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-f6181038-22da-4f97-84a3-7294fa44cf04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 22:00:11.699402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:11.701741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:11.702974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 608/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 22:00:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 22:00:11.710427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a68efe5-c2c2-4cd5-a810-0c46d46b8122 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:11.820343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a68efe5-c2c2-4cd5-a810-0c46d46b8122 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:11.845260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:11.858319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1e195-3cd7-46ae-b101-dead5af7ce11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:11.858941 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1e195-3cd7-46ae-b101-dead5af7ce11 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:11.859219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1e195-3cd7-46ae-b101-dead5af7ce11 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:11.859489 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1e195-3cd7-46ae-b101-dead5af7ce11 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:11.860293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1e195-3cd7-46ae-b101-dead5af7ce11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:11.860505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1e195-3cd7-46ae-b101-dead5af7ce11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:11.861959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1e195-3cd7-46ae-b101-dead5af7ce11 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:11.871205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 615/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 22:00:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:11.877566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6181038-22da-4f97-84a3-7294fa44cf04 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:11.924089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:11.997865 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6181038-22da-4f97-84a3-7294fa44cf04 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:12.037884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76158df8-5e4e-4240-919c-6515f8baaecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:12.038518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76158df8-5e4e-4240-919c-6515f8baaecf None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:12.038842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76158df8-5e4e-4240-919c-6515f8baaecf None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:12.039149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76158df8-5e4e-4240-919c-6515f8baaecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:12.051309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76158df8-5e4e-4240-919c-6515f8baaecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:12.052736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76158df8-5e4e-4240-919c-6515f8baaecf None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-76158df8-5e4e-4240-919c-6515f8baaecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:12.068972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:12.070262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:12.071395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:12.071395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:12.071395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:12.071395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 609/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:12.076177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1e195-3cd7-46ae-b101-dead5af7ce11 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:12.193252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1e195-3cd7-46ae-b101-dead5af7ce11 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:12.236117 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393766f2-cfc7-48d3-9892-86b0f60a0003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:12.237494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393766f2-cfc7-48d3-9892-86b0f60a0003 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:12.237494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393766f2-cfc7-48d3-9892-86b0f60a0003 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:12.237715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393766f2-cfc7-48d3-9892-86b0f60a0003 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:12.238181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393766f2-cfc7-48d3-9892-86b0f60a0003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:12.238477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393766f2-cfc7-48d3-9892-86b0f60a0003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:12.239847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393766f2-cfc7-48d3-9892-86b0f60a0003 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:12.256954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 616/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:12.265553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76158df8-5e4e-4240-919c-6515f8baaecf None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:12.370802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:12.380063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76158df8-5e4e-4240-919c-6515f8baaecf None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:12.418066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffbb4e-b981-43be-b9a3-c878adf84761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:12.419221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffbb4e-b981-43be-b9a3-c878adf84761 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:12.419524 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffbb4e-b981-43be-b9a3-c878adf84761 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:12.419829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffbb4e-b981-43be-b9a3-c878adf84761 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:12.428330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffbb4e-b981-43be-b9a3-c878adf84761 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:12.429998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffbb4e-b981-43be-b9a3-c878adf84761 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-cfffbb4e-b981-43be-b9a3-c878adf84761 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 22:00:12.442348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:12.443595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:12.444756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:12.444756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:12.444756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 22:00:12.444756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 22:00:12.444756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 22:00:12.444756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 22:00:12.444756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 22:00:12.444756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 22:00:12.444756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:12.444756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:12.444756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 610/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:12.449686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393766f2-cfc7-48d3-9892-86b0f60a0003 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:12.500338 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:12.563314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393766f2-cfc7-48d3-9892-86b0f60a0003 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:12.601594 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18f3ca-e47f-44d9-abde-b4805442dbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:12.602205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18f3ca-e47f-44d9-abde-b4805442dbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:12.602480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18f3ca-e47f-44d9-abde-b4805442dbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:12.602753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c18f3ca-e47f-44d9-abde-b4805442dbd1 None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:12.603550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18f3ca-e47f-44d9-abde-b4805442dbd1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:12.604057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18f3ca-e47f-44d9-abde-b4805442dbd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:12.605643 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c18f3ca-e47f-44d9-abde-b4805442dbd1 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:12.625231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 617/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 22 22:00:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:12.631048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffbb4e-b981-43be-b9a3-c878adf84761 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:12.751628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffbb4e-b981-43be-b9a3-c878adf84761 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:12.793171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0abf4f-814f-464a-a5a7-cdf5af10f5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:12.794983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0abf4f-814f-464a-a5a7-cdf5af10f5d0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:12.794983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0abf4f-814f-464a-a5a7-cdf5af10f5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:12.794983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0abf4f-814f-464a-a5a7-cdf5af10f5d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:12.806251 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0abf4f-814f-464a-a5a7-cdf5af10f5d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:12.807702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0abf4f-814f-464a-a5a7-cdf5af10f5d0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-3f0abf4f-814f-464a-a5a7-cdf5af10f5d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:12.822653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:12.823767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:12.824742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:12.824742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:12.824742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:12.824742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 611/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:12.830589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18f3ca-e47f-44d9-abde-b4805442dbd1 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:12.938042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c18f3ca-e47f-44d9-abde-b4805442dbd1 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:12.940628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:12.975497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa529b-c1e6-46e4-bb1f-33dc0675eaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:12.976122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa529b-c1e6-46e4-bb1f-33dc0675eaa0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:12.976413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa529b-c1e6-46e4-bb1f-33dc0675eaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:12.976662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa529b-c1e6-46e4-bb1f-33dc0675eaa0 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:12.977325 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa529b-c1e6-46e4-bb1f-33dc0675eaa0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:12.977624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa529b-c1e6-46e4-bb1f-33dc0675eaa0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:12.979003 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa529b-c1e6-46e4-bb1f-33dc0675eaa0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:12.990709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 618/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:12.996509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0abf4f-814f-464a-a5a7-cdf5af10f5d0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:13.006562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:13.107511 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0abf4f-814f-464a-a5a7-cdf5af10f5d0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:13.143569 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf759b-dbd2-47bd-b997-731fdf0fe106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:13.145757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf759b-dbd2-47bd-b997-731fdf0fe106 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:13.146031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf759b-dbd2-47bd-b997-731fdf0fe106 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:13.146307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf759b-dbd2-47bd-b997-731fdf0fe106 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:13.157019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf759b-dbd2-47bd-b997-731fdf0fe106 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:13.158481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf759b-dbd2-47bd-b997-731fdf0fe106 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-2ebf759b-dbd2-47bd-b997-731fdf0fe106 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 22:00:13.172094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:13.173172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:13.174120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 22:00:13.174120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 22:00:13.174120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 22:00:13.174120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 22:00:13.174120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 22:00:13.174120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 22:00:13.174120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:13.174120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:13.174120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 612/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 22:00:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:13.180013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa529b-c1e6-46e4-bb1f-33dc0675eaa0 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:13.297285 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa529b-c1e6-46e4-bb1f-33dc0675eaa0 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:13.334528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f42e306-2566-48da-87b2-69d8afc0e314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:13.335229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42e306-2566-48da-87b2-69d8afc0e314 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:13.335492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42e306-2566-48da-87b2-69d8afc0e314 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:13.335817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42e306-2566-48da-87b2-69d8afc0e314 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:13.336569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42e306-2566-48da-87b2-69d8afc0e314 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:13.336894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42e306-2566-48da-87b2-69d8afc0e314 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:13.338448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42e306-2566-48da-87b2-69d8afc0e314 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:13.351641 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 619/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 22:00:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:13.358228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf759b-dbd2-47bd-b997-731fdf0fe106 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:13.424429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:13.462735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf759b-dbd2-47bd-b997-731fdf0fe106 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:13.499662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5aac2-4b2f-4c40-802a-217040c5b136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:13.500314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5aac2-4b2f-4c40-802a-217040c5b136 None admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:13.500595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5aac2-4b2f-4c40-802a-217040c5b136 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:13.500894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5aac2-4b2f-4c40-802a-217040c5b136 None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/43d6ba7b68884509a4e2859ceb72ef0e/roles/054c6569cc994d1c950d5e554273033e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:13.516625 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5aac2-4b2f-4c40-802a-217040c5b136 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=43d6ba7b68884509a4e2859ceb72ef0e, role_id=054c6569cc994d1c950d5e554273033e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:13.518087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5aac2-4b2f-4c40-802a-217040c5b136 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:13.530931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 613/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 22:00:13 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/43d6ba7b68884509a4e2859ceb72ef0e/roles/054c6569cc994d1c950d5e554273033e => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:00:15.823799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720e64b-112d-477e-85b7-f3077cd41796 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:15.824413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720e64b-112d-477e-85b7-f3077cd41796 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:15.824413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720e64b-112d-477e-85b7-f3077cd41796 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:15.825927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 620/1233] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:00:15.831415 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a69c5-744b-4c48-a46c-88cb9711f179 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:15.831980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a69c5-744b-4c48-a46c-88cb9711f179 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:15.831980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563a69c5-744b-4c48-a46c-88cb9711f179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:15.852657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-563a69c5-744b-4c48-a46c-88cb9711f179 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:00:15.853879 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-563a69c5-744b-4c48-a46c-88cb9711f179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:00:15.925007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 614/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:00:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 22:00:16.229497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720e64b-112d-477e-85b7-f3077cd41796 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:16.259367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:16.343712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720e64b-112d-477e-85b7-f3077cd41796 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:16.381841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63541fc6-31a9-4fce-b6d0-6ee79a380c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:16.382340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63541fc6-31a9-4fce-b6d0-6ee79a380c32 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:16.382609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63541fc6-31a9-4fce-b6d0-6ee79a380c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:16.382871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63541fc6-31a9-4fce-b6d0-6ee79a380c32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:16.393315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63541fc6-31a9-4fce-b6d0-6ee79a380c32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-63541fc6-31a9-4fce-b6d0-6ee79a380c32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 22:00:16.398114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:16.401317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:16.402652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 621/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Thu Sep 22 22:00:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:16.405712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a69c5-744b-4c48-a46c-88cb9711f179 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:16.457347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:16.517884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563a69c5-744b-4c48-a46c-88cb9711f179 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:16.553837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a3e24-27ec-4cff-a399-491c27618a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:16.554444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3e24-27ec-4cff-a399-491c27618a74 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:16.554726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3e24-27ec-4cff-a399-491c27618a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:16.554997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a3e24-27ec-4cff-a399-491c27618a74 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:16.555646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a3e24-27ec-4cff-a399-491c27618a74 None admin] RBAC: Adding query filter params (name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:16.555948 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a3e24-27ec-4cff-a399-491c27618a74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:16.557339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a3e24-27ec-4cff-a399-491c27618a74 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:16.565544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 615/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Thu Sep 22 22:00:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:16.572170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63541fc6-31a9-4fce-b6d0-6ee79a380c32 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:16.669500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:16.685972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63541fc6-31a9-4fce-b6d0-6ee79a380c32 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:16.726123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305eb9c3-1b0a-4fe9-bd8c-28ae73916df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:16.726811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305eb9c3-1b0a-4fe9-bd8c-28ae73916df3 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:16.727131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305eb9c3-1b0a-4fe9-bd8c-28ae73916df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:16.727458 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305eb9c3-1b0a-4fe9-bd8c-28ae73916df3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:16.739830 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305eb9c3-1b0a-4fe9-bd8c-28ae73916df3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:16.741482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305eb9c3-1b0a-4fe9-bd8c-28ae73916df3 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-305eb9c3-1b0a-4fe9-bd8c-28ae73916df3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:16.759928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:16.761658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:16.763392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:16.763392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:16.763392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:16.763392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 622/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 22:00:16.768033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a3e24-27ec-4cff-a399-491c27618a74 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:16.878356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a3e24-27ec-4cff-a399-491c27618a74 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:16.888412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:16.913029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307f0bc-6393-4371-9995-8c4d593b17eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:16.913654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307f0bc-6393-4371-9995-8c4d593b17eb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:16.913934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307f0bc-6393-4371-9995-8c4d593b17eb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:16.914208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307f0bc-6393-4371-9995-8c4d593b17eb None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:16.914840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307f0bc-6393-4371-9995-8c4d593b17eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:16.915145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307f0bc-6393-4371-9995-8c4d593b17eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:16.916510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307f0bc-6393-4371-9995-8c4d593b17eb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:16.928171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 616/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:16.934573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305eb9c3-1b0a-4fe9-bd8c-28ae73916df3 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:17.048344 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305eb9c3-1b0a-4fe9-bd8c-28ae73916df3 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:17.088681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbef4fe-7e54-4c80-9cb6-71fe6b343d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:17.089344 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbef4fe-7e54-4c80-9cb6-71fe6b343d37 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:17.089623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbef4fe-7e54-4c80-9cb6-71fe6b343d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:17.089894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbef4fe-7e54-4c80-9cb6-71fe6b343d37 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:17.092951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:17.098967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbef4fe-7e54-4c80-9cb6-71fe6b343d37 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:17.100600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbef4fe-7e54-4c80-9cb6-71fe6b343d37 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ccbef4fe-7e54-4c80-9cb6-71fe6b343d37 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 22:00:17.112642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:17.113670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:17.115232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:17.115232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:17.115232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 22:00:17.115232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 22:00:17.115232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 22:00:17.115232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 22:00:17.115232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 22:00:17.115232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 22:00:17.115232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:17.115232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:17.115232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 623/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:17.120131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307f0bc-6393-4371-9995-8c4d593b17eb None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:17.235239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307f0bc-6393-4371-9995-8c4d593b17eb None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:17.274229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e89cc-7693-4d80-aaee-2927b3e0c346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:17.274835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e89cc-7693-4d80-aaee-2927b3e0c346 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:17.275137 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e89cc-7693-4d80-aaee-2927b3e0c346 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:17.275400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4e89cc-7693-4d80-aaee-2927b3e0c346 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:17.276101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e89cc-7693-4d80-aaee-2927b3e0c346 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:17.276404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e89cc-7693-4d80-aaee-2927b3e0c346 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:17.278067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4e89cc-7693-4d80-aaee-2927b3e0c346 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:17.296291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 617/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 22 22:00:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:17.302686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbef4fe-7e54-4c80-9cb6-71fe6b343d37 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:17.416202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbef4fe-7e54-4c80-9cb6-71fe6b343d37 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:17.454039 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c4181-738c-4f75-9578-8441517d786d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:17.454857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c4181-738c-4f75-9578-8441517d786d None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:17.455225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c4181-738c-4f75-9578-8441517d786d None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:17.455509 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c4181-738c-4f75-9578-8441517d786d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:17.525732 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:17.533422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c4181-738c-4f75-9578-8441517d786d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:17.535327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334c4181-738c-4f75-9578-8441517d786d None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:17.658174 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 624/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 22:00:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 22 22:00:17.663935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e89cc-7693-4d80-aaee-2927b3e0c346 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:17.695983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:17.787949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4e89cc-7693-4d80-aaee-2927b3e0c346 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:17.829993 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bde2c8-c4fe-49cf-b745-6cf3d016bca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:17.830631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bde2c8-c4fe-49cf-b745-6cf3d016bca6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:17.830904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bde2c8-c4fe-49cf-b745-6cf3d016bca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:17.831177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bde2c8-c4fe-49cf-b745-6cf3d016bca6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:17.844707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bde2c8-c4fe-49cf-b745-6cf3d016bca6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:17.846206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bde2c8-c4fe-49cf-b745-6cf3d016bca6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-35bde2c8-c4fe-49cf-b745-6cf3d016bca6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:17.866503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:17.867933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:17.869050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:17.869050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:17.869050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:17.869050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 618/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:17.875541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c4181-738c-4f75-9578-8441517d786d None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:17.975950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:17.983290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c4181-738c-4f75-9578-8441517d786d None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:18.021356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9871e7d-8784-4247-94a5-1e91d9010085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:18.022081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871e7d-8784-4247-94a5-1e91d9010085 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:18.022354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871e7d-8784-4247-94a5-1e91d9010085 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:18.022625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9871e7d-8784-4247-94a5-1e91d9010085 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:18.023358 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9871e7d-8784-4247-94a5-1e91d9010085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:18.023696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9871e7d-8784-4247-94a5-1e91d9010085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:18.025326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9871e7d-8784-4247-94a5-1e91d9010085 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:18.039289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 625/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:00:18.045428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bde2c8-c4fe-49cf-b745-6cf3d016bca6 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:18.135560 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:18.159920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bde2c8-c4fe-49cf-b745-6cf3d016bca6 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:18.196194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b92e0-41af-4e1b-904f-153258e39c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:18.196828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b92e0-41af-4e1b-904f-153258e39c4b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:18.197152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b92e0-41af-4e1b-904f-153258e39c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:18.197410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b92e0-41af-4e1b-904f-153258e39c4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:18.205080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b92e0-41af-4e1b-904f-153258e39c4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:18.206617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b92e0-41af-4e1b-904f-153258e39c4b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-1a6b92e0-41af-4e1b-904f-153258e39c4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 22:00:18.219859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:18.220877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:18.222127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 22:00:18.222127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 22:00:18.222127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 22:00:18.222127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 22:00:18.222127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 22:00:18.222127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 22:00:18.222127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:18.222127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:18.222127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 619/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 22:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:18.228198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9871e7d-8784-4247-94a5-1e91d9010085 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:18.348220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9871e7d-8784-4247-94a5-1e91d9010085 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:18.385581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a4afd-0c5a-4024-b678-9613fc804ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:18.386185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a4afd-0c5a-4024-b678-9613fc804ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:18.386463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a4afd-0c5a-4024-b678-9613fc804ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:18.386729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a4afd-0c5a-4024-b678-9613fc804ea0 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:18.388029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a4afd-0c5a-4024-b678-9613fc804ea0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:18.388328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a4afd-0c5a-4024-b678-9613fc804ea0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:18.389790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a4afd-0c5a-4024-b678-9613fc804ea0 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:18.401747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 626/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 22:00:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:18.408187 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b92e0-41af-4e1b-904f-153258e39c4b None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:18.519207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b92e0-41af-4e1b-904f-153258e39c4b None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:18.556172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23e32db-341e-4ca8-8d5e-5324c3879d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:18.557023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e32db-341e-4ca8-8d5e-5324c3879d53 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:18.557327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e32db-341e-4ca8-8d5e-5324c3879d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:18.557595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23e32db-341e-4ca8-8d5e-5324c3879d53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:18.558347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e32db-341e-4ca8-8d5e-5324c3879d53 None admin] RBAC: Adding query filter params (role.id=054c6569cc994d1c950d5e554273033e, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=43d6ba7b68884509a4e2859ceb72ef0e) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:18.558719 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e32db-341e-4ca8-8d5e-5324c3879d53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:18.561097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23e32db-341e-4ca8-8d5e-5324c3879d53 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:18.567594 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 620/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 22:00:18 2022] GET /identity/v3/role_assignments?user.id=43d6ba7b68884509a4e2859ceb72ef0e&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=054c6569cc994d1c950d5e554273033e => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:20.899166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da74df-365b-4dd6-8d43-5746a79a4162 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:20.899752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da74df-365b-4dd6-8d43-5746a79a4162 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:20.899752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da74df-365b-4dd6-8d43-5746a79a4162 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:20.901004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 627/1247] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 22:00:20.907054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96601368-1e36-49e8-a53a-c5b0b06b0a54 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:20.907502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96601368-1e36-49e8-a53a-c5b0b06b0a54 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:20.907606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96601368-1e36-49e8-a53a-c5b0b06b0a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:20.913966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:20.935992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-96601368-1e36-49e8-a53a-c5b0b06b0a54 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:00:20.937404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-96601368-1e36-49e8-a53a-c5b0b06b0a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:00:21.015739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 621/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:00:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 22:00:21.325885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da74df-365b-4dd6-8d43-5746a79a4162 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:21.346733 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:21.441070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da74df-365b-4dd6-8d43-5746a79a4162 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:21.478544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf7092-1140-4851-b414-ae206c66f7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:21.479346 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7092-1140-4851-b414-ae206c66f7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:21.479642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7092-1140-4851-b414-ae206c66f7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:21.480049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf7092-1140-4851-b414-ae206c66f7d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:21.494671 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf7092-1140-4851-b414-ae206c66f7d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-6bdf7092-1140-4851-b414-ae206c66f7d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 22:00:21.500172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:21.501206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:21.502051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 628/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 22:00:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:21.507980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96601368-1e36-49e8-a53a-c5b0b06b0a54 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:21.628130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96601368-1e36-49e8-a53a-c5b0b06b0a54 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:21.661047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:21.666503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984bce4-ce15-43be-9692-2429e59787f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:21.667104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984bce4-ce15-43be-9692-2429e59787f9 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:21.667437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984bce4-ce15-43be-9692-2429e59787f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:21.667750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984bce4-ce15-43be-9692-2429e59787f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:21.669299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984bce4-ce15-43be-9692-2429e59787f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:21.669703 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984bce4-ce15-43be-9692-2429e59787f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:21.671254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984bce4-ce15-43be-9692-2429e59787f9 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:21.680007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 622/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 22:00:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:21.686361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf7092-1140-4851-b414-ae206c66f7d9 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:21.778378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:21.803933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf7092-1140-4851-b414-ae206c66f7d9 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:21.842257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d127f-6cdc-4af9-9e24-8d3906fbfba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:21.842949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d127f-6cdc-4af9-9e24-8d3906fbfba2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:21.843305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d127f-6cdc-4af9-9e24-8d3906fbfba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:21.843584 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8d127f-6cdc-4af9-9e24-8d3906fbfba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:21.855372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d127f-6cdc-4af9-9e24-8d3906fbfba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:21.857842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8d127f-6cdc-4af9-9e24-8d3906fbfba2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ba8d127f-6cdc-4af9-9e24-8d3906fbfba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:21.874688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:21.875847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:21.877196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:21.877196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:21.877196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:21.877196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 629/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:21.882271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984bce4-ce15-43be-9692-2429e59787f9 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:21.992892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984bce4-ce15-43be-9692-2429e59787f9 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:22.028186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aed37a-e8c7-4f92-b96a-421db93ca4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:22.028805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aed37a-e8c7-4f92-b96a-421db93ca4ee None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:22.029081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aed37a-e8c7-4f92-b96a-421db93ca4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:22.029380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aed37a-e8c7-4f92-b96a-421db93ca4ee None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:22.030012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aed37a-e8c7-4f92-b96a-421db93ca4ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:22.030317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aed37a-e8c7-4f92-b96a-421db93ca4ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:22.031662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aed37a-e8c7-4f92-b96a-421db93ca4ee None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:22.044504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 623/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:22.050351 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d127f-6cdc-4af9-9e24-8d3906fbfba2 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:22.169712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8d127f-6cdc-4af9-9e24-8d3906fbfba2 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:22.197711 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:22.207214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e13cb-f356-461e-9cef-819987f36851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:22.208050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e13cb-f356-461e-9cef-819987f36851 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:22.208324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e13cb-f356-461e-9cef-819987f36851 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:22.208590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87e13cb-f356-461e-9cef-819987f36851 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:22.216863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e13cb-f356-461e-9cef-819987f36851 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:22.218421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87e13cb-f356-461e-9cef-819987f36851 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a87e13cb-f356-461e-9cef-819987f36851 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 22:00:22.231373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:22.232898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:22.234025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:22.234025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:22.234025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 22:00:22.234025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 22:00:22.234025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 22:00:22.234025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 22:00:22.234025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 22:00:22.234025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 22:00:22.234025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:22.234025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:22.234025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 630/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:22.239607 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aed37a-e8c7-4f92-b96a-421db93ca4ee None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:22.300838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:22.349449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aed37a-e8c7-4f92-b96a-421db93ca4ee None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:22.385682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f8eec-a97d-4e9f-abaa-1a56d8570810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:22.386295 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f8eec-a97d-4e9f-abaa-1a56d8570810 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:22.386577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f8eec-a97d-4e9f-abaa-1a56d8570810 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:22.386921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0f8eec-a97d-4e9f-abaa-1a56d8570810 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:22.388032 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f8eec-a97d-4e9f-abaa-1a56d8570810 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:22.388337 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f8eec-a97d-4e9f-abaa-1a56d8570810 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:22.389824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0f8eec-a97d-4e9f-abaa-1a56d8570810 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:22.409558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 624/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 22 22:00:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:22.415465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e13cb-f356-461e-9cef-819987f36851 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:22.518447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87e13cb-f356-461e-9cef-819987f36851 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:22.554140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13854115-03c8-4c81-9c54-82922b14cd7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:22.554758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13854115-03c8-4c81-9c54-82922b14cd7c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:22.555032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13854115-03c8-4c81-9c54-82922b14cd7c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:22.555297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13854115-03c8-4c81-9c54-82922b14cd7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:22.613356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:22.626332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13854115-03c8-4c81-9c54-82922b14cd7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:22.627764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13854115-03c8-4c81-9c54-82922b14cd7c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:22.727477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 631/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 22:00:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 22:00:22.734336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f8eec-a97d-4e9f-abaa-1a56d8570810 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:22.850728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0f8eec-a97d-4e9f-abaa-1a56d8570810 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:22.864897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:22.886021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9daff-1f07-47b4-a0b6-a6402f0c84f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:22.887176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9daff-1f07-47b4-a0b6-a6402f0c84f7 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:22.887603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9daff-1f07-47b4-a0b6-a6402f0c84f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:22.887876 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9daff-1f07-47b4-a0b6-a6402f0c84f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:22.898011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9daff-1f07-47b4-a0b6-a6402f0c84f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:22.899457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9daff-1f07-47b4-a0b6-a6402f0c84f7 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-71e9daff-1f07-47b4-a0b6-a6402f0c84f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:22.914690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:22.915794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:22.916910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:22.916910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:22.916910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:22.916910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 625/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:22.923862 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13854115-03c8-4c81-9c54-82922b14cd7c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:23.009883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:23.022484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13854115-03c8-4c81-9c54-82922b14cd7c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:23.056924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77944e66-484e-42bb-8411-abc29bf54972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:23.057584 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77944e66-484e-42bb-8411-abc29bf54972 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:23.057866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77944e66-484e-42bb-8411-abc29bf54972 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:23.058140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77944e66-484e-42bb-8411-abc29bf54972 None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:23.059635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77944e66-484e-42bb-8411-abc29bf54972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:23.059936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77944e66-484e-42bb-8411-abc29bf54972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:23.061324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77944e66-484e-42bb-8411-abc29bf54972 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:23.072633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 632/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:23.078419 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9daff-1f07-47b4-a0b6-a6402f0c84f7 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:23.175128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9daff-1f07-47b4-a0b6-a6402f0c84f7 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:23.208010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800caad6-1d3e-4365-9719-a4373a897466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:23.208615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800caad6-1d3e-4365-9719-a4373a897466 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:23.208892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800caad6-1d3e-4365-9719-a4373a897466 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:23.209186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800caad6-1d3e-4365-9719-a4373a897466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:23.215671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800caad6-1d3e-4365-9719-a4373a897466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:23.217119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800caad6-1d3e-4365-9719-a4373a897466 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-800caad6-1d3e-4365-9719-a4373a897466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 22:00:23.228136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:23.229200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:23.230980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 22:00:23.230980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 22:00:23.230980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 22:00:23.230980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 22:00:23.230980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 22:00:23.230980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 22:00:23.230980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:23.230980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:23.230980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 626/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 22:00:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:23.235642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77944e66-484e-42bb-8411-abc29bf54972 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:23.333596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77944e66-484e-42bb-8411-abc29bf54972 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:23.366099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1650a328-53e6-45a1-9327-00f10bc0d547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:23.366683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650a328-53e6-45a1-9327-00f10bc0d547 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:23.366956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650a328-53e6-45a1-9327-00f10bc0d547 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:23.367221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1650a328-53e6-45a1-9327-00f10bc0d547 None admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:23.367896 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650a328-53e6-45a1-9327-00f10bc0d547 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:23.368194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650a328-53e6-45a1-9327-00f10bc0d547 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:23.369622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1650a328-53e6-45a1-9327-00f10bc0d547 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:23.381018 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 633/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 22:00:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:23.387167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800caad6-1d3e-4365-9719-a4373a897466 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:23.412759 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:23.485699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800caad6-1d3e-4365-9719-a4373a897466 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:23.519214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae57bf7-b357-45fc-8259-383a1f75112b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:23.519806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae57bf7-b357-45fc-8259-383a1f75112b None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:23.520082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae57bf7-b357-45fc-8259-383a1f75112b None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:23.520354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae57bf7-b357-45fc-8259-383a1f75112b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:23.521067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae57bf7-b357-45fc-8259-383a1f75112b None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=43d6ba7b68884509a4e2859ceb72ef0e) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:23.521411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae57bf7-b357-45fc-8259-383a1f75112b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:23.522824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae57bf7-b357-45fc-8259-383a1f75112b None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:23.528075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 627/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 22:00:23 2022] GET /identity/v3/role_assignments?user.id=43d6ba7b68884509a4e2859ceb72ef0e&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:25.647987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853270f-11b8-402c-a3ad-599dcdd7c5b9 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:25.648516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853270f-11b8-402c-a3ad-599dcdd7c5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:25.648516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853270f-11b8-402c-a3ad-599dcdd7c5b9 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:25.649893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 634/1261] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:00:25.655583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737551-6b11-435f-9c77-6459e76a2f1b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:25.655952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737551-6b11-435f-9c77-6459e76a2f1b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:25.656135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44737551-6b11-435f-9c77-6459e76a2f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:25.674741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-44737551-6b11-435f-9c77-6459e76a2f1b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:00:25.675917 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-44737551-6b11-435f-9c77-6459e76a2f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:00:25.743837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 628/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:00:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 22:00:26.023545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853270f-11b8-402c-a3ad-599dcdd7c5b9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:26.034219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:26.122567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853270f-11b8-402c-a3ad-599dcdd7c5b9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:26.154561 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d9b34-2734-40d7-9754-9c9292213c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:26.155190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d9b34-2734-40d7-9754-9c9292213c53 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:26.155462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d9b34-2734-40d7-9754-9c9292213c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:26.155728 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d9b34-2734-40d7-9754-9c9292213c53 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:26.165348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d9b34-2734-40d7-9754-9c9292213c53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-4d9d9b34-2734-40d7-9754-9c9292213c53 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 22:00:26.170508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:26.171608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:26.172505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 635/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 22:00:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:26.177577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44737551-6b11-435f-9c77-6459e76a2f1b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:26.195772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:26.273697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44737551-6b11-435f-9c77-6459e76a2f1b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:26.306996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065169f5-629d-4a33-a310-586244772c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:26.307589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065169f5-629d-4a33-a310-586244772c84 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:26.307865 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065169f5-629d-4a33-a310-586244772c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:26.308135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065169f5-629d-4a33-a310-586244772c84 None admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:26.308835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065169f5-629d-4a33-a310-586244772c84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:26.309149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065169f5-629d-4a33-a310-586244772c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:26.310513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065169f5-629d-4a33-a310-586244772c84 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:26.318189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 629/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 22:00:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:26.323897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d9b34-2734-40d7-9754-9c9292213c53 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:26.387613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:26.419841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d9b34-2734-40d7-9754-9c9292213c53 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:26.452716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf20de1-c58a-4baf-83cf-7c613cd0d324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:26.453353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20de1-c58a-4baf-83cf-7c613cd0d324 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:26.453630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20de1-c58a-4baf-83cf-7c613cd0d324 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:26.453908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf20de1-c58a-4baf-83cf-7c613cd0d324 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:26.463855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf20de1-c58a-4baf-83cf-7c613cd0d324 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:26.466147 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf20de1-c58a-4baf-83cf-7c613cd0d324 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-bcf20de1-c58a-4baf-83cf-7c613cd0d324 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:26.480066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:26.482066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:26.483151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:26.483151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:26.483151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:26.483151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 636/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:26.487911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065169f5-629d-4a33-a310-586244772c84 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:26.565553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:26.584308 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065169f5-629d-4a33-a310-586244772c84 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:26.616626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ebebf-62c0-41f4-bbd7-a3a5aaf927be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:26.617233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ebebf-62c0-41f4-bbd7-a3a5aaf927be None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:26.617507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ebebf-62c0-41f4-bbd7-a3a5aaf927be None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:26.617774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ebebf-62c0-41f4-bbd7-a3a5aaf927be None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:26.618393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ebebf-62c0-41f4-bbd7-a3a5aaf927be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:26.618689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ebebf-62c0-41f4-bbd7-a3a5aaf927be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:26.620034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ebebf-62c0-41f4-bbd7-a3a5aaf927be None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:26.631297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 630/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:26.637828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf20de1-c58a-4baf-83cf-7c613cd0d324 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:26.738675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf20de1-c58a-4baf-83cf-7c613cd0d324 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:26.763070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:26.778220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e3749-2ba4-4f79-bfc4-af0fd753e6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:26.778842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e3749-2ba4-4f79-bfc4-af0fd753e6ed None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:26.779118 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e3749-2ba4-4f79-bfc4-af0fd753e6ed None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:26.779388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069e3749-2ba4-4f79-bfc4-af0fd753e6ed None admin] PATH_INFO: `/v3/users/placement` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:26.786517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e3749-2ba4-4f79-bfc4-af0fd753e6ed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:26.787935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069e3749-2ba4-4f79-bfc4-af0fd753e6ed None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-069e3749-2ba4-4f79-bfc4-af0fd753e6ed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 22:00:26.798690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:26.799821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:26.800897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:26.800897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:26.800897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 22:00:26.800897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 22:00:26.800897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 22:00:26.800897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 22:00:26.800897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 22:00:26.800897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 22:00:26.800897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:26.800897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:26.800897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 637/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:26.805788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ebebf-62c0-41f4-bbd7-a3a5aaf927be None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:26.904777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ebebf-62c0-41f4-bbd7-a3a5aaf927be None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:26.937095 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834d749-04ce-4287-96d8-36dcb20183d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:26.937696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834d749-04ce-4287-96d8-36dcb20183d1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:26.937969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834d749-04ce-4287-96d8-36dcb20183d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:26.938235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834d749-04ce-4287-96d8-36dcb20183d1 None admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:26.938949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834d749-04ce-4287-96d8-36dcb20183d1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:26.939250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834d749-04ce-4287-96d8-36dcb20183d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:26.940654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834d749-04ce-4287-96d8-36dcb20183d1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:26.949447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:26.958499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 631/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 22 22:00:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:26.964402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e3749-2ba4-4f79-bfc4-af0fd753e6ed None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:27.061460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069e3749-2ba4-4f79-bfc4-af0fd753e6ed None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:27.094667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1b03-9e51-4070-a128-fa2fccf6529a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:27.095279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1b03-9e51-4070-a128-fa2fccf6529a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:27.095553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1b03-9e51-4070-a128-fa2fccf6529a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:27.095823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c1b03-9e51-4070-a128-fa2fccf6529a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:27.106028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c1b03-9e51-4070-a128-fa2fccf6529a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:27.107449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c1b03-9e51-4070-a128-fa2fccf6529a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-e74c1b03-9e51-4070-a128-fa2fccf6529a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:27.121516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:27.122519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:27.123901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:27.123901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:27.123901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:27.123901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 638/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:27.128264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834d749-04ce-4287-96d8-36dcb20183d1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:27.224498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834d749-04ce-4287-96d8-36dcb20183d1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:27.257036 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66f6a6-3f6b-4afc-9806-370f1f670ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:27.257658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f6a6-3f6b-4afc-9806-370f1f670ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:27.257935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f6a6-3f6b-4afc-9806-370f1f670ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:27.258206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f6a6-3f6b-4afc-9806-370f1f670ff1 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:27.258837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66f6a6-3f6b-4afc-9806-370f1f670ff1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:27.259136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66f6a6-3f6b-4afc-9806-370f1f670ff1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:27.260525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66f6a6-3f6b-4afc-9806-370f1f670ff1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:27.271967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 632/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:27.277513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1b03-9e51-4070-a128-fa2fccf6529a None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:27.307491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:27.375683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c1b03-9e51-4070-a128-fa2fccf6529a None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:27.407202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e77ec4-3426-456b-8980-51e08a936f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:27.407816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e77ec4-3426-456b-8980-51e08a936f7c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:27.408089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e77ec4-3426-456b-8980-51e08a936f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:27.408362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e77ec4-3426-456b-8980-51e08a936f7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:27.414834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e77ec4-3426-456b-8980-51e08a936f7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:27.416254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e77ec4-3426-456b-8980-51e08a936f7c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-b3e77ec4-3426-456b-8980-51e08a936f7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 22:00:27.427574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:27.428687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:27.429653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 22:00:27.429653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 22:00:27.429653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 22:00:27.429653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 22:00:27.429653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 22:00:27.429653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 22:00:27.429653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:27.429653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:27.429653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 639/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 22:00:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:27.435266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66f6a6-3f6b-4afc-9806-370f1f670ff1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:27.492730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:27.534381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66f6a6-3f6b-4afc-9806-370f1f670ff1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:27.567915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2a290-ae2a-4fe4-a2bd-e99ce5511bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:27.568505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2a290-ae2a-4fe4-a2bd-e99ce5511bdb None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:27.568785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2a290-ae2a-4fe4-a2bd-e99ce5511bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:27.569055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2a290-ae2a-4fe4-a2bd-e99ce5511bdb None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:27.569764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2a290-ae2a-4fe4-a2bd-e99ce5511bdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:27.570070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2a290-ae2a-4fe4-a2bd-e99ce5511bdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:27.571493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2a290-ae2a-4fe4-a2bd-e99ce5511bdb None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:27.583489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 633/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 22:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:27.589473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e77ec4-3426-456b-8980-51e08a936f7c None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:27.679923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:27.688758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e77ec4-3426-456b-8980-51e08a936f7c None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:27.723571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a4c38-afa4-470b-b91e-911513f7a39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:27.724259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a4c38-afa4-470b-b91e-911513f7a39a None admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:27.724535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a4c38-afa4-470b-b91e-911513f7a39a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:27.724840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a4c38-afa4-470b-b91e-911513f7a39a None admin] PATH_INFO: `/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/43d6ba7b68884509a4e2859ceb72ef0e/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:27.726821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a4c38-afa4-470b-b91e-911513f7a39a None admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f546fc57464a8267c5a4f7bde1f8, user_id=43d6ba7b68884509a4e2859ceb72ef0e, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:27.728228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a4c38-afa4-470b-b91e-911513f7a39a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:27.742387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 640/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Thu Sep 22 22:00:27 2022] PUT /identity/v3/projects/d7b0f546fc57464a8267c5a4f7bde1f8/users/43d6ba7b68884509a4e2859ceb72ef0e/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:00:29.868157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af6d1a-952c-419b-8e52-7da2d0190000 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:29.868729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af6d1a-952c-419b-8e52-7da2d0190000 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:29.868729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af6d1a-952c-419b-8e52-7da2d0190000 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:29.870065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 634/1274] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:00:29.875568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc86c5-e91c-4267-ac55-bd33c76a7f10 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:29.876005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc86c5-e91c-4267-ac55-bd33c76a7f10 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:29.876116 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc86c5-e91c-4267-ac55-bd33c76a7f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:29.894372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-44bc86c5-e91c-4267-ac55-bd33c76a7f10 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:00:29.895559 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-44bc86c5-e91c-4267-ac55-bd33c76a7f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:00:29.960079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 641/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:00:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 22:00:30.232708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af6d1a-952c-419b-8e52-7da2d0190000 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:30.331096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af6d1a-952c-419b-8e52-7da2d0190000 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:30.348899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:30.366186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6daea-8aa0-479d-a575-fd6e7ac93290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:30.366800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6daea-8aa0-479d-a575-fd6e7ac93290 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:30.367078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6daea-8aa0-479d-a575-fd6e7ac93290 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:30.367350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6daea-8aa0-479d-a575-fd6e7ac93290 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:30.377777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6daea-8aa0-479d-a575-fd6e7ac93290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-96b6daea-8aa0-479d-a575-fd6e7ac93290 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 22 22:00:30.382436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:30.383550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:30.384786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 635/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Thu Sep 22 22:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:30.389850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc86c5-e91c-4267-ac55-bd33c76a7f10 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:30.470651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:30.489913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc86c5-e91c-4267-ac55-bd33c76a7f10 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:30.523506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d07c9f-b9d2-4d17-9e32-ca4c7ce7b053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:30.524103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07c9f-b9d2-4d17-9e32-ca4c7ce7b053 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:30.524383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07c9f-b9d2-4d17-9e32-ca4c7ce7b053 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:30.524654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d07c9f-b9d2-4d17-9e32-ca4c7ce7b053 None admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:30.525329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d07c9f-b9d2-4d17-9e32-ca4c7ce7b053 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:30.525629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d07c9f-b9d2-4d17-9e32-ca4c7ce7b053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:30.527765 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d07c9f-b9d2-4d17-9e32-ca4c7ce7b053 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:30.535600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 642/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Thu Sep 22 22:00:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:30.541469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6daea-8aa0-479d-a575-fd6e7ac93290 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:30.637741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6daea-8aa0-479d-a575-fd6e7ac93290 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:30.670761 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cadd1-317a-4e85-89a1-d7a7119fd736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:30.671361 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cadd1-317a-4e85-89a1-d7a7119fd736 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:30.671638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cadd1-317a-4e85-89a1-d7a7119fd736 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:30.671911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cadd1-317a-4e85-89a1-d7a7119fd736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:30.681551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cadd1-317a-4e85-89a1-d7a7119fd736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:30.683180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cadd1-317a-4e85-89a1-d7a7119fd736 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-644cadd1-317a-4e85-89a1-d7a7119fd736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:30.697204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:30.698305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:30.699820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:30.699820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:30.699820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:30.699820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 636/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:30.704552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d07c9f-b9d2-4d17-9e32-ca4c7ce7b053 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:30.811429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d07c9f-b9d2-4d17-9e32-ca4c7ce7b053 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:30.843801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3cd34-1c92-495a-9cf2-087ca15bf67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:30.844387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3cd34-1c92-495a-9cf2-087ca15bf67e None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:30.844658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3cd34-1c92-495a-9cf2-087ca15bf67e None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:30.844921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3cd34-1c92-495a-9cf2-087ca15bf67e None admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:30.845572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3cd34-1c92-495a-9cf2-087ca15bf67e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:30.845868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3cd34-1c92-495a-9cf2-087ca15bf67e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:30.847210 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb3cd34-1c92-495a-9cf2-087ca15bf67e None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:30.858302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 643/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:30.863965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cadd1-317a-4e85-89a1-d7a7119fd736 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:30.894488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:30.960327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cadd1-317a-4e85-89a1-d7a7119fd736 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:30.992082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b225c2-755a-4ac9-9c52-fa4f9fc3d6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:30.992686 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b225c2-755a-4ac9-9c52-fa4f9fc3d6af None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:30.992958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b225c2-755a-4ac9-9c52-fa4f9fc3d6af None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:30.993255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b225c2-755a-4ac9-9c52-fa4f9fc3d6af None admin] PATH_INFO: `/v3/users/placement` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:31.000757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b225c2-755a-4ac9-9c52-fa4f9fc3d6af None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:31.002200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b225c2-755a-4ac9-9c52-fa4f9fc3d6af None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-88b225c2-755a-4ac9-9c52-fa4f9fc3d6af None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 22 22:00:31.013402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:31.014521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:31.015510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:31.015510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:31.015510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 22 22:00:31.015510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 22 22:00:31.015510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 22 22:00:31.015510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 22 22:00:31.015510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 22 22:00:31.015510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 22 22:00:31.015510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 22 22:00:31.015510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:31.015510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 637/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:31.020256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3cd34-1c92-495a-9cf2-087ca15bf67e None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:31.024015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:31.116686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3cd34-1c92-495a-9cf2-087ca15bf67e None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:31.149115 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1ad72-e2ac-4c80-b873-7b1242f400ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:31.149714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1ad72-e2ac-4c80-b873-7b1242f400ab None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:31.149989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1ad72-e2ac-4c80-b873-7b1242f400ab None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:31.150257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1ad72-e2ac-4c80-b873-7b1242f400ab None admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:31.150966 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1ad72-e2ac-4c80-b873-7b1242f400ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:31.151265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1ad72-e2ac-4c80-b873-7b1242f400ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:31.152816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1ad72-e2ac-4c80-b873-7b1242f400ab None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:31.169712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 644/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Thu Sep 22 22:00:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:31.175574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b225c2-755a-4ac9-9c52-fa4f9fc3d6af None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:31.263705 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:31.273437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b225c2-755a-4ac9-9c52-fa4f9fc3d6af None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:31.306601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916360e-3b3c-4879-9506-afcc1fbeb3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:31.307198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916360e-3b3c-4879-9506-afcc1fbeb3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:31.307477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916360e-3b3c-4879-9506-afcc1fbeb3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:31.307749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5916360e-3b3c-4879-9506-afcc1fbeb3e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:31.373505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916360e-3b3c-4879-9506-afcc1fbeb3e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:31.374857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5916360e-3b3c-4879-9506-afcc1fbeb3e1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:31.459062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:31.469174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 638/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Thu Sep 22 22:00:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 22 22:00:31.474963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1ad72-e2ac-4c80-b873-7b1242f400ab None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:31.534643 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:31.572903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1ad72-e2ac-4c80-b873-7b1242f400ab None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:31.606400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6a81b-abf2-4a91-9faf-2f7128802b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:31.607012 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6a81b-abf2-4a91-9faf-2f7128802b75 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:31.607287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6a81b-abf2-4a91-9faf-2f7128802b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:31.607552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6a81b-abf2-4a91-9faf-2f7128802b75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:31.617070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6a81b-abf2-4a91-9faf-2f7128802b75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:31.618521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6a81b-abf2-4a91-9faf-2f7128802b75 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-a9a6a81b-abf2-4a91-9faf-2f7128802b75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 22 22:00:31.632554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:31.633677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 22 22:00:31.634797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 22 22:00:31.634797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 22 22:00:31.634797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:31.634797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 645/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:31.639487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916360e-3b3c-4879-9506-afcc1fbeb3e1 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:31.738779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5916360e-3b3c-4879-9506-afcc1fbeb3e1 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:31.772116 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfbbd7-636a-40c4-a6ab-eb262aedc8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:31.772701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfbbd7-636a-40c4-a6ab-eb262aedc8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:31.772978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfbbd7-636a-40c4-a6ab-eb262aedc8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:31.773276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfbbd7-636a-40c4-a6ab-eb262aedc8c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:31.773902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfbbd7-636a-40c4-a6ab-eb262aedc8c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:31.774202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfbbd7-636a-40c4-a6ab-eb262aedc8c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:31.775549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfbbd7-636a-40c4-a6ab-eb262aedc8c0 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:31.786979 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 639/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Thu Sep 22 22:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:31.793262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6a81b-abf2-4a91-9faf-2f7128802b75 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:31.889327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6a81b-abf2-4a91-9faf-2f7128802b75 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:31.902511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:31.922097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba673880-c888-420c-983f-ed115c21edf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:31.922705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba673880-c888-420c-983f-ed115c21edf7 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:31.922978 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba673880-c888-420c-983f-ed115c21edf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:31.923246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba673880-c888-420c-983f-ed115c21edf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:31.929510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba673880-c888-420c-983f-ed115c21edf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:31.930937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba673880-c888-420c-983f-ed115c21edf7 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application [None req-ba673880-c888-420c-983f-ed115c21edf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 22 22:00:31.941807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:31.942885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:31.944004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 22 22:00:31.944004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 22 22:00:31.944004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 22 22:00:31.944004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 22 22:00:31.944004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 22 22:00:31.944004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 22 22:00:31.944004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 22 22:00:31.944004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: ERROR keystone.server.flask.application Sep 22 22:00:31.944004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 646/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Thu Sep 22 22:00:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 22 22:00:31.949448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfbbd7-636a-40c4-a6ab-eb262aedc8c0 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:31.997578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:32.045985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfbbd7-636a-40c4-a6ab-eb262aedc8c0 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:32.078895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140080f5-e6e5-4fd9-a6e0-ca014d77b9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:32.079484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140080f5-e6e5-4fd9-a6e0-ca014d77b9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:32.079756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140080f5-e6e5-4fd9-a6e0-ca014d77b9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:32.080020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140080f5-e6e5-4fd9-a6e0-ca014d77b9d6 None admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:32.080697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140080f5-e6e5-4fd9-a6e0-ca014d77b9d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:32.080996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140080f5-e6e5-4fd9-a6e0-ca014d77b9d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:32.082692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140080f5-e6e5-4fd9-a6e0-ca014d77b9d6 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:32.095048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 640/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Thu Sep 22 22:00:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:32.101208 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba673880-c888-420c-983f-ed115c21edf7 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:32.197792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba673880-c888-420c-983f-ed115c21edf7 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:32.230723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4e6023-19d5-4c36-a617-223f1c50194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:32.231300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e6023-19d5-4c36-a617-223f1c50194c None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:32.231572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e6023-19d5-4c36-a617-223f1c50194c None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:32.231845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4e6023-19d5-4c36-a617-223f1c50194c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:32.232552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e6023-19d5-4c36-a617-223f1c50194c None admin] RBAC: Adding query filter params (role.id=c1595a2afa7947148e6cfe39b0e4990c, scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8, user.id=43d6ba7b68884509a4e2859ceb72ef0e) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:32.232857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e6023-19d5-4c36-a617-223f1c50194c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:32.234298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4e6023-19d5-4c36-a617-223f1c50194c None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:32.239803 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 647/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Thu Sep 22 22:00:32 2022] GET /identity/v3/role_assignments?user.id=43d6ba7b68884509a4e2859ceb72ef0e&scope.project.id=d7b0f546fc57464a8267c5a4f7bde1f8&role.id=c1595a2afa7947148e6cfe39b0e4990c => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:34.415304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57421a6-1834-4187-b63d-ba0d1597fa09 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:34.415809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57421a6-1834-4187-b63d-ba0d1597fa09 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:34.415809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57421a6-1834-4187-b63d-ba0d1597fa09 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:34.417627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 641/1288] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:00:34.423168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5131ee2-301b-4c54-b07a-a88ebf329b41 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:34.423566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5131ee2-301b-4c54-b07a-a88ebf329b41 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:34.423714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5131ee2-301b-4c54-b07a-a88ebf329b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:34.442169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f5131ee2-301b-4c54-b07a-a88ebf329b41 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:00:34.443334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f5131ee2-301b-4c54-b07a-a88ebf329b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:00:34.472275 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:34.513425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 648/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:00:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 22:00:34.782007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57421a6-1834-4187-b63d-ba0d1597fa09 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:34.879831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57421a6-1834-4187-b63d-ba0d1597fa09 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:34.890546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:34.913411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b447e-ab69-4746-9c83-ff52bce62310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:34.914024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b447e-ab69-4746-9c83-ff52bce62310 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:34.914295 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b447e-ab69-4746-9c83-ff52bce62310 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:34.914561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b447e-ab69-4746-9c83-ff52bce62310 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:34.915132 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b447e-ab69-4746-9c83-ff52bce62310 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:34.916467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7b447e-ab69-4746-9c83-ff52bce62310 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-ad7b447e-ab69-4746-9c83-ff52bce62310 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:34.924859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:34.926168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:34.927239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 642/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 22 22:00:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:34.932077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5131ee2-301b-4c54-b07a-a88ebf329b41 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:35.030359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5131ee2-301b-4c54-b07a-a88ebf329b41 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:35.067792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a96b10-375a-4288-8df6-4dac65e6144a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:35.068395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a96b10-375a-4288-8df6-4dac65e6144a None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:35.068673 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a96b10-375a-4288-8df6-4dac65e6144a None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:35.068944 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a96b10-375a-4288-8df6-4dac65e6144a None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:35.069601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a96b10-375a-4288-8df6-4dac65e6144a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:35.069903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a96b10-375a-4288-8df6-4dac65e6144a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:35.071254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a96b10-375a-4288-8df6-4dac65e6144a None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:35.076159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 649/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 22 22:00:34 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:35.082166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b447e-ab69-4746-9c83-ff52bce62310 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:35.177388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b447e-ab69-4746-9c83-ff52bce62310 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:35.209600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ebcc9-20ed-4ebc-993e-8d3682791193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:35.210185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ebcc9-20ed-4ebc-993e-8d3682791193 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:35.210458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ebcc9-20ed-4ebc-993e-8d3682791193 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:35.210726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ebcc9-20ed-4ebc-993e-8d3682791193 None admin] PATH_INFO: `/v3/services` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:35.211336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ebcc9-20ed-4ebc-993e-8d3682791193 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:35.211633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ebcc9-20ed-4ebc-993e-8d3682791193 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:35.212981 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ebcc9-20ed-4ebc-993e-8d3682791193 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:35.217392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 643/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 22 22:00:35 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:37.325972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3518f-4905-499b-9f85-a3695f5329af None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:37.326748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3518f-4905-499b-9f85-a3695f5329af None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:37.326748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3518f-4905-499b-9f85-a3695f5329af None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:37.328337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 650/1293] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:00:37.333740 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa1af1-04c3-42ae-a5d6-bba103e7c260 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:37.334170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa1af1-04c3-42ae-a5d6-bba103e7c260 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:37.334301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aa1af1-04c3-42ae-a5d6-bba103e7c260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:37.352474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-86aa1af1-04c3-42ae-a5d6-bba103e7c260 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:00:37.353675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-86aa1af1-04c3-42ae-a5d6-bba103e7c260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:00:37.381377 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:37.418527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 644/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:00:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 22 22:00:37.685722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3518f-4905-499b-9f85-a3695f5329af None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:37.716429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:37.797713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3518f-4905-499b-9f85-a3695f5329af None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:37.830598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20968020-56f4-4aea-8a85-61e440879f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:37.831194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20968020-56f4-4aea-8a85-61e440879f75 None admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:37.831471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20968020-56f4-4aea-8a85-61e440879f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:37.832009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20968020-56f4-4aea-8a85-61e440879f75 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:37.832635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20968020-56f4-4aea-8a85-61e440879f75 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:37.834006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20968020-56f4-4aea-8a85-61e440879f75 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:37.849115 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 651/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Thu Sep 22 22:00:37 2022] POST /identity/v3/services => generated 246 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:00:39.991279 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52a931-3395-4ff1-9454-13b76442f4f1 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:39.991841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52a931-3395-4ff1-9454-13b76442f4f1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:39.991841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca52a931-3395-4ff1-9454-13b76442f4f1 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:39.993191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 645/1296] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:00:39.998880 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f057-b739-4f4d-abb7-5e0e9cc1db8f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:39.999286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f057-b739-4f4d-abb7-5e0e9cc1db8f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:39.999427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85f057-b739-4f4d-abb7-5e0e9cc1db8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:40.017228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bf85f057-b739-4f4d-abb7-5e0e9cc1db8f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:00:40.018404 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bf85f057-b739-4f4d-abb7-5e0e9cc1db8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:00:40.089313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 652/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:00:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 22:00:40.355818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52a931-3395-4ff1-9454-13b76442f4f1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:40.458423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca52a931-3395-4ff1-9454-13b76442f4f1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:40.492631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f4ec5-25bc-43f2-b5ff-91e374703ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:40.493299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4ec5-25bc-43f2-b5ff-91e374703ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:40.493579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4ec5-25bc-43f2-b5ff-91e374703ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:40.493849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f4ec5-25bc-43f2-b5ff-91e374703ff8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:40.494446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f4ec5-25bc-43f2-b5ff-91e374703ff8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:40.496664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70f4ec5-25bc-43f2-b5ff-91e374703ff8 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-d70f4ec5-25bc-43f2-b5ff-91e374703ff8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:40.505599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:40.506751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:40.508008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 646/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 22 22:00:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:40.513348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85f057-b739-4f4d-abb7-5e0e9cc1db8f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:40.531411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:40.621406 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf85f057-b739-4f4d-abb7-5e0e9cc1db8f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:40.654341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd3a45-6dc1-4fd7-b404-96db90e44d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:40.654933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd3a45-6dc1-4fd7-b404-96db90e44d37 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:40.655209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd3a45-6dc1-4fd7-b404-96db90e44d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:40.655476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd3a45-6dc1-4fd7-b404-96db90e44d37 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:40.656093 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd3a45-6dc1-4fd7-b404-96db90e44d37 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:40.656391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd3a45-6dc1-4fd7-b404-96db90e44d37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:40.657760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dd3a45-6dc1-4fd7-b404-96db90e44d37 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:40.662635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 653/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 22 22:00:40 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:40.669587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f4ec5-25bc-43f2-b5ff-91e374703ff8 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:40.700462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:40.778173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f4ec5-25bc-43f2-b5ff-91e374703ff8 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:40.811836 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3925f-039f-48fe-8054-f1efb6578dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:40.812422 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3925f-039f-48fe-8054-f1efb6578dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:40.812697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3925f-039f-48fe-8054-f1efb6578dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:40.812965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3925f-039f-48fe-8054-f1efb6578dd1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:40.813681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3925f-039f-48fe-8054-f1efb6578dd1 None admin] RBAC: Adding query filter params (interface=public, service_id=7d2a310d92b24dbba2d66d593082da89, region_id=RegionOne) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:40.813984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3925f-039f-48fe-8054-f1efb6578dd1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:40.815330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3925f-039f-48fe-8054-f1efb6578dd1 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:40.820119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 647/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Thu Sep 22 22:00:40 2022] GET /identity/v3/endpoints?service_id=7d2a310d92b24dbba2d66d593082da89&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:40.826148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd3a45-6dc1-4fd7-b404-96db90e44d37 None admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:40.908138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:40.926429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd3a45-6dc1-4fd7-b404-96db90e44d37 None admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:40.959885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a0d436-5b42-4475-844c-b0c0b57c63ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:40.960476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0d436-5b42-4475-844c-b0c0b57c63ad None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:40.960754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0d436-5b42-4475-844c-b0c0b57c63ad None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:40.961022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a0d436-5b42-4475-844c-b0c0b57c63ad None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:40.961647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0d436-5b42-4475-844c-b0c0b57c63ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:40.962989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a0d436-5b42-4475-844c-b0c0b57c63ad None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:40.970528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 654/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Thu Sep 22 22:00:40 2022] GET /identity/v3/services => generated 2191 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:00:43.115169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a16341-958a-4669-a33f-df50533250d7 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:43.115706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a16341-958a-4669-a33f-df50533250d7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:43.115706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a16341-958a-4669-a33f-df50533250d7 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:43.117035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 648/1302] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:00:43.122975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d3477-3200-4e30-8043-a2610636b5d3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:43.123394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d3477-3200-4e30-8043-a2610636b5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:43.123488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61d3477-3200-4e30-8043-a2610636b5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:43.140902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b61d3477-3200-4e30-8043-a2610636b5d3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:00:43.142110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b61d3477-3200-4e30-8043-a2610636b5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:00:43.204960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 655/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:00:43 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 22 22:00:43.481798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a16341-958a-4669-a33f-df50533250d7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:43.573353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:43.580165 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a16341-958a-4669-a33f-df50533250d7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:43.613291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b4340-3e66-4462-912b-4176e991d3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:43.613901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b4340-3e66-4462-912b-4176e991d3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:43.614183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b4340-3e66-4462-912b-4176e991d3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:43.614457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b4340-3e66-4462-912b-4176e991d3e3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:43.615032 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b4340-3e66-4462-912b-4176e991d3e3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:43.616381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b4340-3e66-4462-912b-4176e991d3e3 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application [None req-895b4340-3e66-4462-912b-4176e991d3e3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 22 22:00:43.625407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 22 22:00:43.626539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.get_or_create( Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application with Lock( Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self._enter() Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application return self.creator() Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application created_value = creator( Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: ERROR keystone.server.flask.application Sep 22 22:00:43.627523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 649/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Thu Sep 22 22:00:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 22 22:00:43.632422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d3477-3200-4e30-8043-a2610636b5d3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:43.706795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:43.729321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61d3477-3200-4e30-8043-a2610636b5d3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:43.762523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b6117-e0b5-40bd-bd97-dad7156b79c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:43.763114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b6117-e0b5-40bd-bd97-dad7156b79c2 None admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:43.763389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b6117-e0b5-40bd-bd97-dad7156b79c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:43.763665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b6117-e0b5-40bd-bd97-dad7156b79c2 None admin] PATH_INFO: `/v3/services` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:43.764279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b6117-e0b5-40bd-bd97-dad7156b79c2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:00:43.764577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b6117-e0b5-40bd-bd97-dad7156b79c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:43.765985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9b6117-e0b5-40bd-bd97-dad7156b79c2 None admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:43.770888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 656/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Thu Sep 22 22:00:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:00:43.777277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b4340-3e66-4462-912b-4176e991d3e3 None admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:00:43.873032 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b4340-3e66-4462-912b-4176e991d3e3 None admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:00:43.905283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea9a87-ea85-4e99-8cbc-f48b59d18419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:00:43.905864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea9a87-ea85-4e99-8cbc-f48b59d18419 None admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:00:43.906138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea9a87-ea85-4e99-8cbc-f48b59d18419 None admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:00:43.906406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea9a87-ea85-4e99-8cbc-f48b59d18419 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:00:43.907067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea9a87-ea85-4e99-8cbc-f48b59d18419 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:00:43.908401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea9a87-ea85-4e99-8cbc-f48b59d18419 None admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:00:43.926977 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 650/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:00:43 2022] POST /identity/v3/endpoints => generated 331 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:00:53.768025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=30 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:00:53.922690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:01:48.192928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a799b-c502-4b1f-9631-be9b151c29f3 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:48.193992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a799b-c502-4b1f-9631-be9b151c29f3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:48.194393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a799b-c502-4b1f-9631-be9b151c29f3 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:48.197532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 657/1307] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:01:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:01:48.200864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16950a-4a38-44f9-a7c7-67a3e9b9e189 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:48.201321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16950a-4a38-44f9-a7c7-67a3e9b9e189 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:48.201441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c16950a-4a38-44f9-a7c7-67a3e9b9e189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:48.289625 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7c16950a-4a38-44f9-a7c7-67a3e9b9e189 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:01:48.291184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7c16950a-4a38-44f9-a7c7-67a3e9b9e189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:48.400682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 651/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:01:48.699418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d544d9-d3ff-4564-99f8-56b1dfa97ac1 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:48.699701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d544d9-d3ff-4564-99f8-56b1dfa97ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:48.700262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d544d9-d3ff-4564-99f8-56b1dfa97ac1 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:48.701716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 658/1309] 10.222.0.25 () {54 vars in 964 bytes} [Thu Sep 22 22:01:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:01:48.708476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8cf9f-3413-4a3a-9b36-b8d0a822af4e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:48.709458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8cf9f-3413-4a3a-9b36-b8d0a822af4e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:48.709841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8cf9f-3413-4a3a-9b36-b8d0a822af4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:48.796971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b0d8cf9f-3413-4a3a-9b36-b8d0a822af4e None None] MFA Rules not processed for user `18c956fa035f40fa88fbf662541121ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:01:48.798347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b0d8cf9f-3413-4a3a-9b36-b8d0a822af4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:48.821776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:01:48.881180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 652/1310] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Sep 22 22:01:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:01:48.890276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d544d9-d3ff-4564-99f8-56b1dfa97ac1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:01:48.892148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-13d544d9-d3ff-4564-99f8-56b1dfa97ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:49.016313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d544d9-d3ff-4564-99f8-56b1dfa97ac1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:01:49.104213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fbf7e-338c-4bc2-8620-25866ad292c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:01:49.104969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fbf7e-338c-4bc2-8620-25866ad292c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:49.105267 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fbf7e-338c-4bc2-8620-25866ad292c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:49.105534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fbf7e-338c-4bc2-8620-25866ad292c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:49.107104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-038fbf7e-338c-4bc2-8620-25866ad292c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:49.167486 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:01:49.227739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fbf7e-338c-4bc2-8620-25866ad292c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:01:49.229332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fbf7e-338c-4bc2-8620-25866ad292c8 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:01:49.249508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 659/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:01:51.787429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb006f-6a08-49d2-b824-6c165772b79e None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:51.788058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb006f-6a08-49d2-b824-6c165772b79e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:51.788058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb006f-6a08-49d2-b824-6c165772b79e None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:51.789346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 653/1312] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:01:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:01:51.794482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1680fb-cf2d-4d56-bd48-03aeaa6d1f6b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:51.794867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1680fb-cf2d-4d56-bd48-03aeaa6d1f6b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:51.795015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1680fb-cf2d-4d56-bd48-03aeaa6d1f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:51.876492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5f1680fb-cf2d-4d56-bd48-03aeaa6d1f6b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:01:51.877731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5f1680fb-cf2d-4d56-bd48-03aeaa6d1f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:51.973418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 660/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:01:52.254844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb006f-6a08-49d2-b824-6c165772b79e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:01:52.257212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-05bb006f-6a08-49d2-b824-6c165772b79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:52.363897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb006f-6a08-49d2-b824-6c165772b79e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:01:52.434804 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:01:52.437693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d8978-e73c-4405-a5b1-34d3396c0c96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:01:52.438484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d8978-e73c-4405-a5b1-34d3396c0c96 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:52.438929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d8978-e73c-4405-a5b1-34d3396c0c96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:52.439391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d8978-e73c-4405-a5b1-34d3396c0c96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:52.440983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0a1d8978-e73c-4405-a5b1-34d3396c0c96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:52.551748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d8978-e73c-4405-a5b1-34d3396c0c96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:01:52.553220 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1d8978-e73c-4405-a5b1-34d3396c0c96 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:01:52.571132 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 654/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:01:55.069556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb5e6e-7d56-4ded-a34b-ff394c0cfaf2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:55.070053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb5e6e-7d56-4ded-a34b-ff394c0cfaf2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:55.070150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb5e6e-7d56-4ded-a34b-ff394c0cfaf2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:55.071448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 661/1315] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:01:55.076649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a439db5-4a56-470f-95ee-225fb555872f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:55.077027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a439db5-4a56-470f-95ee-225fb555872f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:55.077239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a439db5-4a56-470f-95ee-225fb555872f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:55.141631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5a439db5-4a56-470f-95ee-225fb555872f None None] MFA Rules not processed for user `68d47cfe578c4b96a86d50549ee2c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:01:55.142837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5a439db5-4a56-470f-95ee-225fb555872f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:55.193550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:01:55.237326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 655/1316] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:01:55 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:01:55.519032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb5e6e-7d56-4ded-a34b-ff394c0cfaf2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:01:55.520576 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e5cb5e6e-7d56-4ded-a34b-ff394c0cfaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:55.530287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:01:55.622299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb5e6e-7d56-4ded-a34b-ff394c0cfaf2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:01:55.693548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde58ad-459d-4a6a-8dfc-0ad7d4cc0b58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:01:55.694071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde58ad-459d-4a6a-8dfc-0ad7d4cc0b58 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:55.694349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde58ad-459d-4a6a-8dfc-0ad7d4cc0b58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:55.694616 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde58ad-459d-4a6a-8dfc-0ad7d4cc0b58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:55.696172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7fde58ad-459d-4a6a-8dfc-0ad7d4cc0b58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:55.758645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:01:55.805568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde58ad-459d-4a6a-8dfc-0ad7d4cc0b58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:01:55.806992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde58ad-459d-4a6a-8dfc-0ad7d4cc0b58 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:01:55.824704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 662/1317] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:01:55 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:01:58.501444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a787d8-c095-459d-8182-87c5e92191e6 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:58.501961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a787d8-c095-459d-8182-87c5e92191e6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:58.501961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a787d8-c095-459d-8182-87c5e92191e6 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:58.503233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 656/1318] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:01:58.508570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52d52d-d045-416d-8d81-43e4ca7666e6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:58.508997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52d52d-d045-416d-8d81-43e4ca7666e6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:58.509138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52d52d-d045-416d-8d81-43e4ca7666e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:58.573116 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3b52d52d-d045-416d-8d81-43e4ca7666e6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:01:58.574353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3b52d52d-d045-416d-8d81-43e4ca7666e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:58.668147 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 663/1319] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:01:58 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:01:58.956190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a787d8-c095-459d-8182-87c5e92191e6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:01:58.957737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-14a787d8-c095-459d-8182-87c5e92191e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:59.060798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a787d8-c095-459d-8182-87c5e92191e6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:01:59.132739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe800e-1a3e-4d00-9813-1702f1ef6996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:01:59.133356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe800e-1a3e-4d00-9813-1702f1ef6996 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:01:59.133630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe800e-1a3e-4d00-9813-1702f1ef6996 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:01:59.133892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe800e-1a3e-4d00-9813-1702f1ef6996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:01:59.135448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e7fe800e-1a3e-4d00-9813-1702f1ef6996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:01:59.142142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:01:59.243736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe800e-1a3e-4d00-9813-1702f1ef6996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:01:59.245068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe800e-1a3e-4d00-9813-1702f1ef6996 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:01:59.263709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 657/1320] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:01:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:02.054856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe37d-fbd3-479c-a269-bb8f9a87c7ab None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:02.054856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe37d-fbd3-479c-a269-bb8f9a87c7ab None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:02.054856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bafe37d-fbd3-479c-a269-bb8f9a87c7ab None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:02.056548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 664/1321] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:02.060406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dc3ff-d697-438d-a698-8db8e4d11ea6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:02.060795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dc3ff-d697-438d-a698-8db8e4d11ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:02.061077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308dc3ff-d697-438d-a698-8db8e4d11ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:02.133462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-308dc3ff-d697-438d-a698-8db8e4d11ea6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:02.134708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-308dc3ff-d697-438d-a698-8db8e4d11ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:02.196003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:02.239483 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 658/1322] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:02.526579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafe37d-fbd3-479c-a269-bb8f9a87c7ab None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:02.528105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2bafe37d-fbd3-479c-a269-bb8f9a87c7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:02.548379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:02.634346 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bafe37d-fbd3-479c-a269-bb8f9a87c7ab None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:02.709196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fc58c-e1ee-4144-b354-5782232ea90d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:02.709792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fc58c-e1ee-4144-b354-5782232ea90d service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:02.710068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fc58c-e1ee-4144-b354-5782232ea90d service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:02.710341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fc58c-e1ee-4144-b354-5782232ea90d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:02.711919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a17fc58c-e1ee-4144-b354-5782232ea90d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:02.784425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:02.822772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fc58c-e1ee-4144-b354-5782232ea90d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:02.824191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fc58c-e1ee-4144-b354-5782232ea90d service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:02.842402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 665/1323] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:05.078447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e2229-de3e-42fc-9a41-abc1251a9db6 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:05.078997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e2229-de3e-42fc-9a41-abc1251a9db6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:05.078997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e2229-de3e-42fc-9a41-abc1251a9db6 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:05.080318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 659/1324] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:05.085913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61a9fd-de76-4fdd-9e91-e0060f82d710 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:05.086324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61a9fd-de76-4fdd-9e91-e0060f82d710 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:05.086461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc61a9fd-de76-4fdd-9e91-e0060f82d710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:05.152879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fc61a9fd-de76-4fdd-9e91-e0060f82d710 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:05.154115 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fc61a9fd-de76-4fdd-9e91-e0060f82d710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:05.270890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 666/1325] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:05.560478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e2229-de3e-42fc-9a41-abc1251a9db6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:05.562089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d96e2229-de3e-42fc-9a41-abc1251a9db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:05.664841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e2229-de3e-42fc-9a41-abc1251a9db6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:05.740401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f9c3db-29ac-4fbc-8f9f-0eea6f57fdec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:05.741001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9c3db-29ac-4fbc-8f9f-0eea6f57fdec service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:05.741303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9c3db-29ac-4fbc-8f9f-0eea6f57fdec service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:05.741574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f9c3db-29ac-4fbc-8f9f-0eea6f57fdec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:05.743488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-61f9c3db-29ac-4fbc-8f9f-0eea6f57fdec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:05.761233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:05.854867 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9c3db-29ac-4fbc-8f9f-0eea6f57fdec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:05.856206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f9c3db-29ac-4fbc-8f9f-0eea6f57fdec service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:05.884125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 660/1326] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:08.110547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041f14c-be26-4b7c-9705-b16681a94f39 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:08.111034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041f14c-be26-4b7c-9705-b16681a94f39 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:08.111034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041f14c-be26-4b7c-9705-b16681a94f39 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:08.112741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 667/1327] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:08.118742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd3169-d5d2-4b5e-ab54-bc687de9d585 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:08.119185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd3169-d5d2-4b5e-ab54-bc687de9d585 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:08.119359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd3169-d5d2-4b5e-ab54-bc687de9d585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:08.185454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f3cd3169-d5d2-4b5e-ab54-bc687de9d585 None None] MFA Rules not processed for user `68d47cfe578c4b96a86d50549ee2c4ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:08.186673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f3cd3169-d5d2-4b5e-ab54-bc687de9d585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:08.253083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:08.285050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 661/1328] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:02:08.571470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041f14c-be26-4b7c-9705-b16681a94f39 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:08.572996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8041f14c-be26-4b7c-9705-b16681a94f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:08.606344 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:08.681669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8041f14c-be26-4b7c-9705-b16681a94f39 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:08.756238 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c0e7f-0502-4446-82eb-2b2063960f0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:08.756849 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c0e7f-0502-4446-82eb-2b2063960f0f service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:08.757147 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c0e7f-0502-4446-82eb-2b2063960f0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:08.757416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c0e7f-0502-4446-82eb-2b2063960f0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:08.758976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f38c0e7f-0502-4446-82eb-2b2063960f0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:08.839128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:08.870760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c0e7f-0502-4446-82eb-2b2063960f0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:08.872186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c0e7f-0502-4446-82eb-2b2063960f0f service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:08.890492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 668/1329] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:08 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:11.286950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5841db6-3eef-4fe9-9e70-e8e55b405df3 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:11.287515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5841db6-3eef-4fe9-9e70-e8e55b405df3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:11.287515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5841db6-3eef-4fe9-9e70-e8e55b405df3 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:11.288831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 662/1330] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:11.294142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3f05f-c5ba-4056-a5a1-0a7f20b51e4f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:11.294526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3f05f-c5ba-4056-a5a1-0a7f20b51e4f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:11.294689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3f05f-c5ba-4056-a5a1-0a7f20b51e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:11.359416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a6d3f05f-c5ba-4056-a5a1-0a7f20b51e4f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:11.360550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a6d3f05f-c5ba-4056-a5a1-0a7f20b51e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:11.458232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 669/1331] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:11.744331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5841db6-3eef-4fe9-9e70-e8e55b405df3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:11.745885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c5841db6-3eef-4fe9-9e70-e8e55b405df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:11.846293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5841db6-3eef-4fe9-9e70-e8e55b405df3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:11.917440 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a777c4d2-4abd-4928-a776-0348853f86d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:11.918029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777c4d2-4abd-4928-a776-0348853f86d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:11.918301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777c4d2-4abd-4928-a776-0348853f86d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:11.918566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a777c4d2-4abd-4928-a776-0348853f86d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:11.920107 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a777c4d2-4abd-4928-a776-0348853f86d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:11.947727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:12.031147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777c4d2-4abd-4928-a776-0348853f86d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:12.032581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a777c4d2-4abd-4928-a776-0348853f86d9 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:12.051216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 663/1332] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:14.244798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df714502-c6d9-49eb-99ef-47c2e16a31d7 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:14.245364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df714502-c6d9-49eb-99ef-47c2e16a31d7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:14.245364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df714502-c6d9-49eb-99ef-47c2e16a31d7 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:14.246701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 670/1333] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:14.251961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb542a8-e443-4a5c-a550-d3a35caaee36 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:14.252349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb542a8-e443-4a5c-a550-d3a35caaee36 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:14.252535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb542a8-e443-4a5c-a550-d3a35caaee36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:14.319183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-abb542a8-e443-4a5c-a550-d3a35caaee36 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:14.320352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-abb542a8-e443-4a5c-a550-d3a35caaee36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:14.393831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:14.415664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 664/1334] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:14.701534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df714502-c6d9-49eb-99ef-47c2e16a31d7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:14.703303 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-df714502-c6d9-49eb-99ef-47c2e16a31d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:14.743117 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:14.806226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df714502-c6d9-49eb-99ef-47c2e16a31d7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:14.879455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0810c8-ccfd-4475-bb8b-7bddc8577142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:14.880051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0810c8-ccfd-4475-bb8b-7bddc8577142 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:14.880326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0810c8-ccfd-4475-bb8b-7bddc8577142 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:14.880593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0810c8-ccfd-4475-bb8b-7bddc8577142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:14.882187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bb0810c8-ccfd-4475-bb8b-7bddc8577142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:14.963928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:14.989714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0810c8-ccfd-4475-bb8b-7bddc8577142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:14.991142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0810c8-ccfd-4475-bb8b-7bddc8577142 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:15.008557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 671/1335] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:17.486959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c3e7b-b5f4-425f-94ac-2d8bd269dec0 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:17.487533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c3e7b-b5f4-425f-94ac-2d8bd269dec0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:17.487533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c3e7b-b5f4-425f-94ac-2d8bd269dec0 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:17.489418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 665/1336] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 22:02:17.494502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2f00c-b95a-426b-aa41-3f4625d74594 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:17.494935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2f00c-b95a-426b-aa41-3f4625d74594 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:17.495052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2f00c-b95a-426b-aa41-3f4625d74594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:17.560382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-92a2f00c-b95a-426b-aa41-3f4625d74594 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:17.561637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-92a2f00c-b95a-426b-aa41-3f4625d74594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:17.655740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 672/1337] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:17.943411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c3e7b-b5f4-425f-94ac-2d8bd269dec0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:17.944952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f42c3e7b-b5f4-425f-94ac-2d8bd269dec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:18.045648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c3e7b-b5f4-425f-94ac-2d8bd269dec0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:18.119219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca55b3-46ea-44dc-831c-2f5840915854 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:18.119849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca55b3-46ea-44dc-831c-2f5840915854 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:18.120125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca55b3-46ea-44dc-831c-2f5840915854 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:18.120421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca55b3-46ea-44dc-831c-2f5840915854 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:18.122029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dcca55b3-46ea-44dc-831c-2f5840915854 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:18.157690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:18.231615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca55b3-46ea-44dc-831c-2f5840915854 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:18.233025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca55b3-46ea-44dc-831c-2f5840915854 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:18.252143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 666/1338] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:22.358367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4b37f-1570-40f1-8fe5-d123e4acb28b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:22.358947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4b37f-1570-40f1-8fe5-d123e4acb28b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:22.358947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4b37f-1570-40f1-8fe5-d123e4acb28b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:22.360401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 673/1339] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:22.365905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab27a0-f99a-4d17-b4a7-ba8015372dfb None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:22.366354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab27a0-f99a-4d17-b4a7-ba8015372dfb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:22.366469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab27a0-f99a-4d17-b4a7-ba8015372dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:22.437754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d7ab27a0-f99a-4d17-b4a7-ba8015372dfb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:22.438980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d7ab27a0-f99a-4d17-b4a7-ba8015372dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:22.543149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:22.555857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 667/1340] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:22.841768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4b37f-1570-40f1-8fe5-d123e4acb28b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:22.843336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-51a4b37f-1570-40f1-8fe5-d123e4acb28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:22.893592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:22.948237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4b37f-1570-40f1-8fe5-d123e4acb28b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:23.021902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbdaebf-48f9-4f04-a9a3-3e1fd5d9367c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:23.022505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdaebf-48f9-4f04-a9a3-3e1fd5d9367c service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:23.022778 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdaebf-48f9-4f04-a9a3-3e1fd5d9367c service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:23.023045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbdaebf-48f9-4f04-a9a3-3e1fd5d9367c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:23.024600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bdbdaebf-48f9-4f04-a9a3-3e1fd5d9367c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:23.131116 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:23.148466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdaebf-48f9-4f04-a9a3-3e1fd5d9367c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:23.149912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbdaebf-48f9-4f04-a9a3-3e1fd5d9367c service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:23.167587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 674/1341] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:26.004812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177c317-4e36-4849-9161-0a13de0c1784 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:26.005421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177c317-4e36-4849-9161-0a13de0c1784 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:26.005421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177c317-4e36-4849-9161-0a13de0c1784 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:26.006929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 668/1342] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:26.011903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9f434-2107-456c-8e8a-f928421fad27 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:26.012353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9f434-2107-456c-8e8a-f928421fad27 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:26.012586 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9f434-2107-456c-8e8a-f928421fad27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:26.083346 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-abe9f434-2107-456c-8e8a-f928421fad27 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:26.084576 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-abe9f434-2107-456c-8e8a-f928421fad27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:26.185682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 675/1343] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:26.480826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177c317-4e36-4849-9161-0a13de0c1784 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:26.482386 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5177c317-4e36-4849-9161-0a13de0c1784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:26.584063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177c317-4e36-4849-9161-0a13de0c1784 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:26.659877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd93a08-e4fa-4566-94ab-935913475888 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:26.660395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93a08-e4fa-4566-94ab-935913475888 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:26.660672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93a08-e4fa-4566-94ab-935913475888 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:26.660938 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd93a08-e4fa-4566-94ab-935913475888 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:26.662550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7dd93a08-e4fa-4566-94ab-935913475888 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:26.707648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:26.779893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd93a08-e4fa-4566-94ab-935913475888 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:26.781358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd93a08-e4fa-4566-94ab-935913475888 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:26.800342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 669/1344] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:30.941755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4b230-4486-4586-8330-a4acbfbd16aa None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:30.942295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4b230-4486-4586-8330-a4acbfbd16aa None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:30.942295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc4b230-4486-4586-8330-a4acbfbd16aa None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:30.943926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 676/1345] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:30.949391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918dfd0-c190-492b-8a93-514c9eab3110 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:30.949855 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918dfd0-c190-492b-8a93-514c9eab3110 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:30.949963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e918dfd0-c190-492b-8a93-514c9eab3110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:31.013950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e918dfd0-c190-492b-8a93-514c9eab3110 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:31.015155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e918dfd0-c190-492b-8a93-514c9eab3110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:31.110450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:31.113031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 670/1346] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:31.396304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4b230-4486-4586-8330-a4acbfbd16aa None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:31.397885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8bc4b230-4486-4586-8330-a4acbfbd16aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:31.460809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:31.509692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc4b230-4486-4586-8330-a4acbfbd16aa None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:31.588523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042ce50-0f34-4475-b728-a3a893e25c46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:31.589163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042ce50-0f34-4475-b728-a3a893e25c46 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:31.589446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042ce50-0f34-4475-b728-a3a893e25c46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:31.589720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042ce50-0f34-4475-b728-a3a893e25c46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:31.591415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c042ce50-0f34-4475-b728-a3a893e25c46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:31.706635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:31.711047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042ce50-0f34-4475-b728-a3a893e25c46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:31.712512 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042ce50-0f34-4475-b728-a3a893e25c46 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:31.730766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 677/1347] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:36.619580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c828e-ad14-42e5-834b-af6e2b5d133b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:36.620075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c828e-ad14-42e5-834b-af6e2b5d133b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:36.620075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c828e-ad14-42e5-834b-af6e2b5d133b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:36.621721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 671/1348] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:36.626689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558bafd-0c04-45db-ab18-66e7f02de2bc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:36.627698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558bafd-0c04-45db-ab18-66e7f02de2bc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:36.627787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558bafd-0c04-45db-ab18-66e7f02de2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:36.746923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4558bafd-0c04-45db-ab18-66e7f02de2bc None None] MFA Rules not processed for user `43d6ba7b68884509a4e2859ceb72ef0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:36.748514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4558bafd-0c04-45db-ab18-66e7f02de2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:36.911497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 678/1349] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:02:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:41.068941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119f6c-b19b-48d4-936c-8c8b1fab8ac5 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:41.069488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119f6c-b19b-48d4-936c-8c8b1fab8ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:41.069488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119f6c-b19b-48d4-936c-8c8b1fab8ac5 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:41.071020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 672/1350] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 22:02:41.077382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed702fae-1d52-4ba8-838f-8ae142cc9d57 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:41.077868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed702fae-1d52-4ba8-838f-8ae142cc9d57 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:41.078136 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed702fae-1d52-4ba8-838f-8ae142cc9d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:41.135696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:41.160744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ed702fae-1d52-4ba8-838f-8ae142cc9d57 None None] MFA Rules not processed for user `43d6ba7b68884509a4e2859ceb72ef0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:41.162100 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ed702fae-1d52-4ba8-838f-8ae142cc9d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:41.275016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 679/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:02:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:42.605503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79b365-2717-4551-a966-1654989e7d5c None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:42.606079 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79b365-2717-4551-a966-1654989e7d5c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:42.606079 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c79b365-2717-4551-a966-1654989e7d5c None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:42.607402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 673/1352] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:42.612956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30f19f-b75a-4156-a0ad-387b0d64fa77 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:42.613417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30f19f-b75a-4156-a0ad-387b0d64fa77 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:42.613646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30f19f-b75a-4156-a0ad-387b0d64fa77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:42.688212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ef30f19f-b75a-4156-a0ad-387b0d64fa77 None None] MFA Rules not processed for user `43d6ba7b68884509a4e2859ceb72ef0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:42.690112 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ef30f19f-b75a-4156-a0ad-387b0d64fa77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:42.734461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:42.799564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 680/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:02:42 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:46.461210 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5ff0e-e533-492e-8b3f-c16da898beac None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:46.461883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5ff0e-e533-492e-8b3f-c16da898beac None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:46.461883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd5ff0e-e533-492e-8b3f-c16da898beac None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:46.463094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 674/1354] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:46.470203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ec2f-8e57-4e4b-b32c-3576af9dd7e4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:46.471171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ec2f-8e57-4e4b-b32c-3576af9dd7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:46.471431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d1ec2f-8e57-4e4b-b32c-3576af9dd7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:46.549871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-39d1ec2f-8e57-4e4b-b32c-3576af9dd7e4 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:46.551597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-39d1ec2f-8e57-4e4b-b32c-3576af9dd7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:46.668555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 681/1355] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:46.996141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79faa0-f3a9-411c-9e0f-521f9221d1aa None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:46.996431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79faa0-f3a9-411c-9e0f-521f9221d1aa None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:46.996731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79faa0-f3a9-411c-9e0f-521f9221d1aa None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:46.998254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 675/1356] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 22 22:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:47.014247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a0fe1-ae08-4774-8b28-8b750c1a0074 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:47.014756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a0fe1-ae08-4774-8b28-8b750c1a0074 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:47.016600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014a0fe1-ae08-4774-8b28-8b750c1a0074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:47.042769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:47.131949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-014a0fe1-ae08-4774-8b28-8b750c1a0074 None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:47.133324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-014a0fe1-ae08-4774-8b28-8b750c1a0074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:47.264790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 682/1357] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 22 22:02:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:47.273272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79faa0-f3a9-411c-9e0f-521f9221d1aa None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:47.274883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8d79faa0-f3a9-411c-9e0f-521f9221d1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:47.430890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79faa0-f3a9-411c-9e0f-521f9221d1aa None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:47.511491 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:47.557381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07b4598-d1ae-4c8a-8210-96cd7e876e17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:47.558058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b4598-d1ae-4c8a-8210-96cd7e876e17 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:47.558426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b4598-d1ae-4c8a-8210-96cd7e876e17 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:47.558806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07b4598-d1ae-4c8a-8210-96cd7e876e17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:47.560457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d07b4598-d1ae-4c8a-8210-96cd7e876e17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:47.656459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b4598-d1ae-4c8a-8210-96cd7e876e17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:47.658292 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07b4598-d1ae-4c8a-8210-96cd7e876e17 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:47.676467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 676/1358] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:50.072829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b385f-ed73-4ef2-a648-7f2c81d6c35e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:50.073617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b385f-ed73-4ef2-a648-7f2c81d6c35e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:50.073617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b385f-ed73-4ef2-a648-7f2c81d6c35e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:50.078947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 683/1359] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:50.080259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e9ec4-5a88-4e5d-a8c7-5316107d7db7 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:50.080659 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e9ec4-5a88-4e5d-a8c7-5316107d7db7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:50.080804 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4e9ec4-5a88-4e5d-a8c7-5316107d7db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:50.153899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-bf4e9ec4-5a88-4e5d-a8c7-5316107d7db7 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:50.155184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bf4e9ec4-5a88-4e5d-a8c7-5316107d7db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:50.194022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:50.259097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 677/1360] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:50.547655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b385f-ed73-4ef2-a648-7f2c81d6c35e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:50.549971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-843b385f-ed73-4ef2-a648-7f2c81d6c35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:50.629825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:50.734008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b385f-ed73-4ef2-a648-7f2c81d6c35e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:50.861718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0b4962-7aee-4db5-b83a-aa782634fcd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:50.862332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b4962-7aee-4db5-b83a-aa782634fcd4 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:50.862610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b4962-7aee-4db5-b83a-aa782634fcd4 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:50.862877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0b4962-7aee-4db5-b83a-aa782634fcd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:50.864478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5c0b4962-7aee-4db5-b83a-aa782634fcd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:50.887090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:50.958237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b4962-7aee-4db5-b83a-aa782634fcd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:50.959890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0b4962-7aee-4db5-b83a-aa782634fcd4 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:50.978713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 684/1361] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:53.493015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d205-01dd-4b68-a9af-12a227c42edf None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:53.493562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d205-01dd-4b68-a9af-12a227c42edf None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:53.493643 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86d205-01dd-4b68-a9af-12a227c42edf None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:53.494929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 678/1362] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:53.500145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5118b-6863-4657-ae83-fbc955fccfd8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:53.500549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5118b-6863-4657-ae83-fbc955fccfd8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:53.500726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5118b-6863-4657-ae83-fbc955fccfd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:53.581514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-74d5118b-6863-4657-ae83-fbc955fccfd8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:53.582905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-74d5118b-6863-4657-ae83-fbc955fccfd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:53.690276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:53.697217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 685/1363] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:53.983077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb6699-1b62-4710-b925-32d699ac4606 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:53.983471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb6699-1b62-4710-b925-32d699ac4606 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:53.983849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcb6699-1b62-4710-b925-32d699ac4606 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:53.985546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 679/1364] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 22 22:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:53.990894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec651d-64e8-404a-9901-f6e68f24369c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:53.991353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec651d-64e8-404a-9901-f6e68f24369c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:53.991640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec651d-64e8-404a-9901-f6e68f24369c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:54.077366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0cec651d-64e8-404a-9901-f6e68f24369c None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:54.078563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0cec651d-64e8-404a-9901-f6e68f24369c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:54.207688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 686/1365] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 22 22:02:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:54.215356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb6699-1b62-4710-b925-32d699ac4606 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:54.217088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7bcb6699-1b62-4710-b925-32d699ac4606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:54.379319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcb6699-1b62-4710-b925-32d699ac4606 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:54.401673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:54.503266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104bcee-c5d9-4a4b-b24a-9a0024d6e5b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:54.503961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104bcee-c5d9-4a4b-b24a-9a0024d6e5b8 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:54.504233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104bcee-c5d9-4a4b-b24a-9a0024d6e5b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:54.504405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104bcee-c5d9-4a4b-b24a-9a0024d6e5b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:54.506151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8104bcee-c5d9-4a4b-b24a-9a0024d6e5b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:54.606480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104bcee-c5d9-4a4b-b24a-9a0024d6e5b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:54.608077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104bcee-c5d9-4a4b-b24a-9a0024d6e5b8 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:54.627336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 680/1366] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:02:54.635350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee338f5-0bc3-41a2-bb7b-e0e0f0c032bc None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:54.635749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee338f5-0bc3-41a2-bb7b-e0e0f0c032bc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:54.635904 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee338f5-0bc3-41a2-bb7b-e0e0f0c032bc None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:54.637419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 687/1367] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:54.643554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17133d85-017a-49f4-a48c-ec47cdcb6057 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:54.643841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17133d85-017a-49f4-a48c-ec47cdcb6057 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:54.644114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17133d85-017a-49f4-a48c-ec47cdcb6057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:54.678372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:54.717205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-17133d85-017a-49f4-a48c-ec47cdcb6057 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:54.718413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-17133d85-017a-49f4-a48c-ec47cdcb6057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:54.853681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 681/1368] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:02:55.185138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad721f-970e-4913-b529-0587e7f358f0 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:55.185423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad721f-970e-4913-b529-0587e7f358f0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:55.185694 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ad721f-970e-4913-b529-0587e7f358f0 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:55.189148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 688/1369] 10.222.0.25 () {54 vars in 963 bytes} [Thu Sep 22 22:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:55.194135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d72a4-2665-41b1-a3e9-471964301c0a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:55.194429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d72a4-2665-41b1-a3e9-471964301c0a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:55.194700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d72a4-2665-41b1-a3e9-471964301c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:55.287089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-266d72a4-2665-41b1-a3e9-471964301c0a None None] MFA Rules not processed for user `fcfdbb6b3dd04478ae99681cc649fdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:55.288289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-266d72a4-2665-41b1-a3e9-471964301c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:55.338909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:55.372155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 682/1370] 10.222.0.25 () {58 vars in 1060 bytes} [Thu Sep 22 22:02:55 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:02:55.380654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad721f-970e-4913-b529-0587e7f358f0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:55.382360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e2ad721f-970e-4913-b529-0587e7f358f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:55.435978 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:55.496360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ad721f-970e-4913-b529-0587e7f358f0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:55.580961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dea0b2-622d-4366-93c3-2774d36fc1b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:55.581614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dea0b2-622d-4366-93c3-2774d36fc1b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:55.581893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dea0b2-622d-4366-93c3-2774d36fc1b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:55.582171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dea0b2-622d-4366-93c3-2774d36fc1b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:55.583820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-63dea0b2-622d-4366-93c3-2774d36fc1b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:55.703023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:55.719105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dea0b2-622d-4366-93c3-2774d36fc1b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:55.721411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dea0b2-622d-4366-93c3-2774d36fc1b5 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:55.745491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 689/1371] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:56.869872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07cf6-d7c7-4e0b-b10b-d02bc05e902f None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:56.870441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07cf6-d7c7-4e0b-b10b-d02bc05e902f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:56.870441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f07cf6-d7c7-4e0b-b10b-d02bc05e902f None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:56.871766 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 683/1372] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:56.877356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b5ebe-52c7-458b-8c1c-50b1dc3a8f3a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:56.877788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b5ebe-52c7-458b-8c1c-50b1dc3a8f3a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:56.878097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b5ebe-52c7-458b-8c1c-50b1dc3a8f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:56.944255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d56b5ebe-52c7-458b-8c1c-50b1dc3a8f3a None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:02:56.945511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d56b5ebe-52c7-458b-8c1c-50b1dc3a8f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:57.040927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 690/1373] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:02:57.336367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f07cf6-d7c7-4e0b-b10b-d02bc05e902f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:02:57.338179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-06f07cf6-d7c7-4e0b-b10b-d02bc05e902f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:57.490784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f07cf6-d7c7-4e0b-b10b-d02bc05e902f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:02:57.543957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:02:57.607902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26627541-2ebf-4f9f-8be5-34d2c2ca0572 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:02:57.608505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26627541-2ebf-4f9f-8be5-34d2c2ca0572 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:57.608779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26627541-2ebf-4f9f-8be5-34d2c2ca0572 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:57.609045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26627541-2ebf-4f9f-8be5-34d2c2ca0572 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:57.610698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-26627541-2ebf-4f9f-8be5-34d2c2ca0572 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:02:57.705460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26627541-2ebf-4f9f-8be5-34d2c2ca0572 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:02:57.707436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26627541-2ebf-4f9f-8be5-34d2c2ca0572 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:02:57.726462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 684/1374] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:02:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:02:59.931961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d2ec5-ada5-4d3c-9c89-5df224b3b845 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:59.932543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d2ec5-ada5-4d3c-9c89-5df224b3b845 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:59.932543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d2ec5-ada5-4d3c-9c89-5df224b3b845 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:02:59.934035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 691/1375] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:02:59.939825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339eedc-ce5c-4427-85d5-c76c702f6fb2 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:02:59.940219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339eedc-ce5c-4427-85d5-c76c702f6fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:02:59.940402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9339eedc-ce5c-4427-85d5-c76c702f6fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:00.015020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9339eedc-ce5c-4427-85d5-c76c702f6fb2 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:00.016517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9339eedc-ce5c-4427-85d5-c76c702f6fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:00.024027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:00.137571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 685/1376] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:00.429249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d2ec5-ada5-4d3c-9c89-5df224b3b845 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:00.430741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-029d2ec5-ada5-4d3c-9c89-5df224b3b845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:00.494814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:00.584416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d2ec5-ada5-4d3c-9c89-5df224b3b845 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:00.708692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85eeab-e0ab-4a16-acb0-86da4c5a45d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:00.709257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85eeab-e0ab-4a16-acb0-86da4c5a45d0 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:00.709536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85eeab-e0ab-4a16-acb0-86da4c5a45d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:00.709805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85eeab-e0ab-4a16-acb0-86da4c5a45d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:00.711370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ea85eeab-e0ab-4a16-acb0-86da4c5a45d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:00.731230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:00.810799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85eeab-e0ab-4a16-acb0-86da4c5a45d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:00.812388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea85eeab-e0ab-4a16-acb0-86da4c5a45d0 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:00.832817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 692/1377] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:03.070872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea84105-1d30-4b52-ac19-4bfb03d96851 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:03.071365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea84105-1d30-4b52-ac19-4bfb03d96851 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:03.071504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea84105-1d30-4b52-ac19-4bfb03d96851 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:03.072960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 686/1378] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:03.078302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f8810-eb09-4779-9e42-fd7a9f986558 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:03.078752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f8810-eb09-4779-9e42-fd7a9f986558 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:03.078850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f8810-eb09-4779-9e42-fd7a9f986558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:03.149711 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-207f8810-eb09-4779-9e42-fd7a9f986558 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:03.150980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-207f8810-eb09-4779-9e42-fd7a9f986558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:03.263163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:03.277379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 693/1379] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:03.598806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea84105-1d30-4b52-ac19-4bfb03d96851 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:03.600321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9ea84105-1d30-4b52-ac19-4bfb03d96851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:03.752806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:03.764029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea84105-1d30-4b52-ac19-4bfb03d96851 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:03.885350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2083d3-cc04-406c-97c3-6db3e67b9593 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:03.885956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2083d3-cc04-406c-97c3-6db3e67b9593 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:03.886195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2083d3-cc04-406c-97c3-6db3e67b9593 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:03.886461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2083d3-cc04-406c-97c3-6db3e67b9593 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:03.888025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ba2083d3-cc04-406c-97c3-6db3e67b9593 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:03.993609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2083d3-cc04-406c-97c3-6db3e67b9593 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:03.995741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2083d3-cc04-406c-97c3-6db3e67b9593 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:04.008377 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:04.018724 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 687/1380] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:06.283682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8a2b9-59cd-4392-845b-bc2284098505 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:06.284225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8a2b9-59cd-4392-845b-bc2284098505 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:06.284225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8a2b9-59cd-4392-845b-bc2284098505 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:06.285607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 694/1381] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:03:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 22:03:06.292310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cc005-5f4c-4dc0-90a0-6b9e22cb4c5a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:06.292755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cc005-5f4c-4dc0-90a0-6b9e22cb4c5a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:06.292923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4cc005-5f4c-4dc0-90a0-6b9e22cb4c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:06.361202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0f4cc005-5f4c-4dc0-90a0-6b9e22cb4c5a None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:06.362413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0f4cc005-5f4c-4dc0-90a0-6b9e22cb4c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:06.472063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 688/1382] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:06.760220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8a2b9-59cd-4392-845b-bc2284098505 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:06.761797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-52e8a2b9-59cd-4392-845b-bc2284098505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:06.915199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8a2b9-59cd-4392-845b-bc2284098505 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:06.987527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:07.035014 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc950d-20fa-458a-9213-9a614addccac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:07.035626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc950d-20fa-458a-9213-9a614addccac service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:07.035908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc950d-20fa-458a-9213-9a614addccac service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:07.036176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc950d-20fa-458a-9213-9a614addccac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:07.037799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-50fc950d-20fa-458a-9213-9a614addccac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:07.129659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc950d-20fa-458a-9213-9a614addccac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:07.131257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc950d-20fa-458a-9213-9a614addccac service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:07.149741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 695/1383] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:09.364474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21060cec-e227-43c4-add3-eb820dd64d52 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:09.365012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21060cec-e227-43c4-add3-eb820dd64d52 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:09.365012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21060cec-e227-43c4-add3-eb820dd64d52 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:09.366380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 689/1384] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 22:03:09.372046 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a7d9-f9b2-4db4-b1f7-14c61b6321e6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:09.372472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a7d9-f9b2-4db4-b1f7-14c61b6321e6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:09.372587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a7d9-f9b2-4db4-b1f7-14c61b6321e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:09.441317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1718a7d9-f9b2-4db4-b1f7-14c61b6321e6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:09.442528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1718a7d9-f9b2-4db4-b1f7-14c61b6321e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:09.476016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:09.539335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 696/1385] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:09.828849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21060cec-e227-43c4-add3-eb820dd64d52 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:09.830394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-21060cec-e227-43c4-add3-eb820dd64d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:09.907741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:09.977340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21060cec-e227-43c4-add3-eb820dd64d52 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:10.107207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3149802e-e65f-4dca-8adf-06237a6296c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:10.107737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149802e-e65f-4dca-8adf-06237a6296c1 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:10.108013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149802e-e65f-4dca-8adf-06237a6296c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:10.108279 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3149802e-e65f-4dca-8adf-06237a6296c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:10.109944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3149802e-e65f-4dca-8adf-06237a6296c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:10.154442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:10.211160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149802e-e65f-4dca-8adf-06237a6296c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:10.212735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3149802e-e65f-4dca-8adf-06237a6296c1 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:10.231731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 690/1386] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:11.066068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf399d-053b-4d42-aaed-dce07becd1ff None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:11.066549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf399d-053b-4d42-aaed-dce07becd1ff None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:11.066651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf399d-053b-4d42-aaed-dce07becd1ff None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:11.067955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 697/1387] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:11.073262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14739fad-1eb1-4b51-9251-7486bdbd9433 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:11.073656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14739fad-1eb1-4b51-9251-7486bdbd9433 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:11.073837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14739fad-1eb1-4b51-9251-7486bdbd9433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:11.145240 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-14739fad-1eb1-4b51-9251-7486bdbd9433 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:11.146478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-14739fad-1eb1-4b51-9251-7486bdbd9433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:11.243552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 691/1388] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:11.545141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e53dd-52c2-4565-9775-e7b577a2b166 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:11.545456 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e53dd-52c2-4565-9775-e7b577a2b166 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:11.545730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e53dd-52c2-4565-9775-e7b577a2b166 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:11.547661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 698/1389] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 22 22:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:11.553008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3f55b-7ccb-43ba-bf32-a2a82bd31a87 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:11.553326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3f55b-7ccb-43ba-bf32-a2a82bd31a87 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:11.553603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3f55b-7ccb-43ba-bf32-a2a82bd31a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:11.571787 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:11.634498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-60b3f55b-7ccb-43ba-bf32-a2a82bd31a87 None None] MFA Rules not processed for user `195565a7695b4d9b82e0ee76181fee2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:11.635720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-60b3f55b-7ccb-43ba-bf32-a2a82bd31a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:11.711634 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 692/1390] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 22 22:03:11 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:11.721134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e53dd-52c2-4565-9775-e7b577a2b166 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:11.722763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-861e53dd-52c2-4565-9775-e7b577a2b166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:11.827270 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e53dd-52c2-4565-9775-e7b577a2b166 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:11.886908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:11.902863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c2509a-c752-4055-a893-b3c74c916e5b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '195565a7695b4d9b82e0ee76181fee2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:11.903503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2509a-c752-4055-a893-b3c74c916e5b service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:11.903783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2509a-c752-4055-a893-b3c74c916e5b service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:11.904056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c2509a-c752-4055-a893-b3c74c916e5b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:11.905655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-84c2509a-c752-4055-a893-b3c74c916e5b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:12.017838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2509a-c752-4055-a893-b3c74c916e5b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:12.019266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c2509a-c752-4055-a893-b3c74c916e5b service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:12.038307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 699/1391] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:12.476266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c88057-b64c-45f0-a2eb-1dfd2a622da8 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:12.476803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c88057-b64c-45f0-a2eb-1dfd2a622da8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:12.476803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c88057-b64c-45f0-a2eb-1dfd2a622da8 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:12.478191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 693/1392] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:12.483623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992afefb-5b91-4fb5-a693-2f6a5aa007a8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:12.484408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992afefb-5b91-4fb5-a693-2f6a5aa007a8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:12.484408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992afefb-5b91-4fb5-a693-2f6a5aa007a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:12.555722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-992afefb-5b91-4fb5-a693-2f6a5aa007a8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:12.556977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-992afefb-5b91-4fb5-a693-2f6a5aa007a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:12.601577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:12.668248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 700/1393] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:03:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:12.807569 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d592b0e-72d3-466d-8a5c-78fb438bf48a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:12.807883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d592b0e-72d3-466d-8a5c-78fb438bf48a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:12.808160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d592b0e-72d3-466d-8a5c-78fb438bf48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:12.874389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6d592b0e-72d3-466d-8a5c-78fb438bf48a None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:12.876197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6d592b0e-72d3-466d-8a5c-78fb438bf48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:12.928999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:12.961045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 694/1394] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:03:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:12.970038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992afefb-5b91-4fb5-a693-2f6a5aa007a8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:12.970791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-992afefb-5b91-4fb5-a693-2f6a5aa007a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:13.007810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac8e71-86ed-47e4-8514-ea1cd8781214 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:13.008090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac8e71-86ed-47e4-8514-ea1cd8781214 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:13.008362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ac8e71-86ed-47e4-8514-ea1cd8781214 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:13.009731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 695/1395] 10.222.0.25 () {54 vars in 943 bytes} [Thu Sep 22 22:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:13.015714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032024b-a400-498c-85a1-824029ff9c2c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:13.015997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032024b-a400-498c-85a1-824029ff9c2c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:13.016297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e032024b-a400-498c-85a1-824029ff9c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:13.094236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e032024b-a400-498c-85a1-824029ff9c2c None None] MFA Rules not processed for user `471e7550ae7a4f909851215d3ed148c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:13.095590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e032024b-a400-498c-85a1-824029ff9c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:13.100065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992afefb-5b91-4fb5-a693-2f6a5aa007a8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:13.156434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:13.205675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 696/1396] 10.222.0.25 () {58 vars in 1040 bytes} [Thu Sep 22 22:03:13 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:03:13.214996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032024b-a400-498c-85a1-824029ff9c2c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:13.215156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f258dd-7356-4e3f-9d02-4fe2c488b4d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:13.215766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f258dd-7356-4e3f-9d02-4fe2c488b4d4 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:13.216040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f258dd-7356-4e3f-9d02-4fe2c488b4d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:13.216337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f258dd-7356-4e3f-9d02-4fe2c488b4d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:13.217979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-05f258dd-7356-4e3f-9d02-4fe2c488b4d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:13.224843 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e032024b-a400-498c-85a1-824029ff9c2c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:13.250895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:13.309427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f258dd-7356-4e3f-9d02-4fe2c488b4d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:13.311011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f258dd-7356-4e3f-9d02-4fe2c488b4d4 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:13.329418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 701/1397] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:13.330066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a802dd0a-a5a6-4c35-9a30-e52fd0d9469f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:13.330670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dd0a-a5a6-4c35-9a30-e52fd0d9469f service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:13.330949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dd0a-a5a6-4c35-9a30-e52fd0d9469f service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:13.331218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a802dd0a-a5a6-4c35-9a30-e52fd0d9469f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:13.332842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a802dd0a-a5a6-4c35-9a30-e52fd0d9469f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:13.424915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802dd0a-a5a6-4c35-9a30-e52fd0d9469f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:13.426583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a802dd0a-a5a6-4c35-9a30-e52fd0d9469f service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:13.435972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 697/1398] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:03:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 22:03:15.549586 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af56cde-5125-4992-85d8-499f2a024282 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:15.550260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af56cde-5125-4992-85d8-499f2a024282 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:15.550260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af56cde-5125-4992-85d8-499f2a024282 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:15.551487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 702/1399] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:15.556894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21710e74-80f5-441b-bd9c-b1342d6a0ade None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:15.557803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21710e74-80f5-441b-bd9c-b1342d6a0ade None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:15.558088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21710e74-80f5-441b-bd9c-b1342d6a0ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:15.633765 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-21710e74-80f5-441b-bd9c-b1342d6a0ade None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:15.635291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-21710e74-80f5-441b-bd9c-b1342d6a0ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:15.644059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:15.759618 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 698/1400] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:16.070254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af56cde-5125-4992-85d8-499f2a024282 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:16.072055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8af56cde-5125-4992-85d8-499f2a024282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:16.146594 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:16.234489 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af56cde-5125-4992-85d8-499f2a024282 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:16.359337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f83c8c-e0ad-47b0-83cf-38c5593e23ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:16.359860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f83c8c-e0ad-47b0-83cf-38c5593e23ec service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:16.360138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f83c8c-e0ad-47b0-83cf-38c5593e23ec service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:16.360410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f83c8c-e0ad-47b0-83cf-38c5593e23ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:16.362059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e5f83c8c-e0ad-47b0-83cf-38c5593e23ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:16.392146 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:16.459270 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f83c8c-e0ad-47b0-83cf-38c5593e23ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:16.461377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f83c8c-e0ad-47b0-83cf-38c5593e23ec service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:16.479914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 703/1401] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:18.695038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0576d-be7a-4898-b523-0b74dedc539e None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:18.695626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0576d-be7a-4898-b523-0b74dedc539e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:18.695626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f0576d-be7a-4898-b523-0b74dedc539e None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:18.697053 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 699/1402] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:03:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:18.703184 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f56112-3894-44a8-af48-3856adea6480 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:18.703612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f56112-3894-44a8-af48-3856adea6480 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:18.703729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f56112-3894-44a8-af48-3856adea6480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:18.782797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-46f56112-3894-44a8-af48-3856adea6480 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:18.784839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-46f56112-3894-44a8-af48-3856adea6480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:18.891354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 704/1403] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:19.184661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0576d-be7a-4898-b523-0b74dedc539e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:19.186203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-05f0576d-be7a-4898-b523-0b74dedc539e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:19.342872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:19.354692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f0576d-be7a-4898-b523-0b74dedc539e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:19.491616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ed9cef-cb94-4fb7-a6f1-c05addedae8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:19.492221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed9cef-cb94-4fb7-a6f1-c05addedae8a service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:19.492501 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed9cef-cb94-4fb7-a6f1-c05addedae8a service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:19.492774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ed9cef-cb94-4fb7-a6f1-c05addedae8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:19.494389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c2ed9cef-cb94-4fb7-a6f1-c05addedae8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:19.596065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed9cef-cb94-4fb7-a6f1-c05addedae8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:19.597752 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ed9cef-cb94-4fb7-a6f1-c05addedae8a service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:19.608846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:19.629303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 700/1404] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:21.904569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e4aca-f18f-4c88-aa52-498bb159f77e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:21.906471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e4aca-f18f-4c88-aa52-498bb159f77e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:21.907007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e4aca-f18f-4c88-aa52-498bb159f77e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:21.908864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 705/1405] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:03:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:21.914230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dc6b9-9a0c-48a1-8898-3de00ef01101 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:21.914679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dc6b9-9a0c-48a1-8898-3de00ef01101 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:21.914973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89dc6b9-9a0c-48a1-8898-3de00ef01101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:21.987109 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d89dc6b9-9a0c-48a1-8898-3de00ef01101 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:21.988437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d89dc6b9-9a0c-48a1-8898-3de00ef01101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:22.103784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 701/1406] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:03:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:22.404836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e4aca-f18f-4c88-aa52-498bb159f77e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:22.406298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e29e4aca-f18f-4c88-aa52-498bb159f77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:22.409313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:22.570975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29e4aca-f18f-4c88-aa52-498bb159f77e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:22.658981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:22.717271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46af9c5-ca2a-4251-bcdd-2539d696edb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:22.719222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46af9c5-ca2a-4251-bcdd-2539d696edb7 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:22.719827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46af9c5-ca2a-4251-bcdd-2539d696edb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:22.720400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46af9c5-ca2a-4251-bcdd-2539d696edb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:22.724655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b46af9c5-ca2a-4251-bcdd-2539d696edb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:22.840079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46af9c5-ca2a-4251-bcdd-2539d696edb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:22.841992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46af9c5-ca2a-4251-bcdd-2539d696edb7 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:22.862741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 706/1407] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:32.102177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=67 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:32.860000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=61 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:36.604194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c624b9e-3f2a-44ea-a25d-a663410ed04b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:36.604780 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c624b9e-3f2a-44ea-a25d-a663410ed04b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:36.604780 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c624b9e-3f2a-44ea-a25d-a663410ed04b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:36.606141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 702/1408] 10.222.0.25 () {54 vars in 918 bytes} [Thu Sep 22 22:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:36.611695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a359f05-6206-4642-800d-f2c66faf1245 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:36.612131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a359f05-6206-4642-800d-f2c66faf1245 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:36.612248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a359f05-6206-4642-800d-f2c66faf1245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:36.686841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4a359f05-6206-4642-800d-f2c66faf1245 None None] MFA Rules not processed for user `43d6ba7b68884509a4e2859ceb72ef0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:36.688033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4a359f05-6206-4642-800d-f2c66faf1245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:36.791397 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 707/1409] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 22 22:03:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:36.806110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8ce7-1901-4dce-a433-807d190ef765 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:36.806403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8ce7-1901-4dce-a433-807d190ef765 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:36.806677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a8ce7-1901-4dce-a433-807d190ef765 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:36.808054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 703/1410] 10.222.0.25 () {54 vars in 918 bytes} [Thu Sep 22 22:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:36.813521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2616a-8cac-40e9-b6ec-e25979d082be None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:36.813829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2616a-8cac-40e9-b6ec-e25979d082be None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:36.814104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc2616a-8cac-40e9-b6ec-e25979d082be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:36.888399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:36.891794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6cc2616a-8cac-40e9-b6ec-e25979d082be None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:36.893546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6cc2616a-8cac-40e9-b6ec-e25979d082be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:37.007631 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 708/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Thu Sep 22 22:03:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:39.706569 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e1e14-7da2-4b92-b43f-c14cac2afb34 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:39.707128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e1e14-7da2-4b92-b43f-c14cac2afb34 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:39.707128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e1e14-7da2-4b92-b43f-c14cac2afb34 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:39.708502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 704/1412] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:03:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:39.713638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43800816-b78c-4ed9-8aa9-bf43ea36303b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:39.714083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43800816-b78c-4ed9-8aa9-bf43ea36303b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:39.714182 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43800816-b78c-4ed9-8aa9-bf43ea36303b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:39.783609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-43800816-b78c-4ed9-8aa9-bf43ea36303b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:39.784825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-43800816-b78c-4ed9-8aa9-bf43ea36303b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:39.838766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:39.879955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 709/1413] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:40.174839 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e1e14-7da2-4b92-b43f-c14cac2afb34 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:40.176376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c78e1e14-7da2-4b92-b43f-c14cac2afb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:40.324881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e1e14-7da2-4b92-b43f-c14cac2afb34 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:40.403344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:40.443470 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadaabc-0ce1-4da0-a64c-3f4820d6ff0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:40.444069 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadaabc-0ce1-4da0-a64c-3f4820d6ff0d service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:40.444347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadaabc-0ce1-4da0-a64c-3f4820d6ff0d service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:40.444618 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadaabc-0ce1-4da0-a64c-3f4820d6ff0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:40.446243 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-aaadaabc-0ce1-4da0-a64c-3f4820d6ff0d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:40.534141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadaabc-0ce1-4da0-a64c-3f4820d6ff0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:40.535711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadaabc-0ce1-4da0-a64c-3f4820d6ff0d service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:40.553719 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 705/1414] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:41.723551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb581ca-d9a5-4eac-972a-e4c295029094 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:41.724082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb581ca-d9a5-4eac-972a-e4c295029094 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:41.724082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb581ca-d9a5-4eac-972a-e4c295029094 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:41.725454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 710/1415] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:41.731608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbeca-506b-4a81-b645-65774034b97c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:41.732730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbeca-506b-4a81-b645-65774034b97c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:41.732900 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcdbeca-506b-4a81-b645-65774034b97c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:41.808887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1fcdbeca-506b-4a81-b645-65774034b97c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:41.810479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1fcdbeca-506b-4a81-b645-65774034b97c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:41.837314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:41.906931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 706/1416] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:42.057248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb581ca-d9a5-4eac-972a-e4c295029094 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:42.058550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9fb581ca-d9a5-4eac-972a-e4c295029094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:42.206037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb581ca-d9a5-4eac-972a-e4c295029094 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:42.246618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:42.319842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd4039f-b3fb-42a7-ab40-60ef63c33bed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:42.320374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4039f-b3fb-42a7-ab40-60ef63c33bed service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:42.320651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4039f-b3fb-42a7-ab40-60ef63c33bed service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:42.320923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4039f-b3fb-42a7-ab40-60ef63c33bed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:42.322548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5fd4039f-b3fb-42a7-ab40-60ef63c33bed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:42.413774 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4039f-b3fb-42a7-ab40-60ef63c33bed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:42.415349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd4039f-b3fb-42a7-ab40-60ef63c33bed service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:42.432827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 711/1417] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:43.572813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a8612-41b6-4969-9b35-38d0a236f0dc None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:43.573374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a8612-41b6-4969-9b35-38d0a236f0dc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:43.573374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2a8612-41b6-4969-9b35-38d0a236f0dc None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:43.575009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 707/1418] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:43.581440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69eabc-567e-4075-a07f-d83977e38f79 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:43.581876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69eabc-567e-4075-a07f-d83977e38f79 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:43.582003 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69eabc-567e-4075-a07f-d83977e38f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:43.634957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:43.657217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ef69eabc-567e-4075-a07f-d83977e38f79 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:43.658424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ef69eabc-567e-4075-a07f-d83977e38f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:43.757948 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 712/1419] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:43.906725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a8612-41b6-4969-9b35-38d0a236f0dc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:43.908217 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ce2a8612-41b6-4969-9b35-38d0a236f0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:44.169674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2a8612-41b6-4969-9b35-38d0a236f0dc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:44.182162 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:44.282924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9e286-2163-417f-810e-6fac2b72e9c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:44.283522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9e286-2163-417f-810e-6fac2b72e9c3 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:44.283798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9e286-2163-417f-810e-6fac2b72e9c3 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:44.284065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9e286-2163-417f-810e-6fac2b72e9c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:44.285660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cff9e286-2163-417f-810e-6fac2b72e9c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:44.372084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9e286-2163-417f-810e-6fac2b72e9c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:44.373694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9e286-2163-417f-810e-6fac2b72e9c3 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:44.391487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 708/1420] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:45.504805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb504fcc-f775-4825-96ba-54f91d03eb8c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:45.505430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb504fcc-f775-4825-96ba-54f91d03eb8c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:45.505430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb504fcc-f775-4825-96ba-54f91d03eb8c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:45.506765 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 713/1421] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:03:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:45.513500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf767634-7f77-4cce-bd7a-9308fd4dfe0f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:45.513996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf767634-7f77-4cce-bd7a-9308fd4dfe0f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:45.514111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf767634-7f77-4cce-bd7a-9308fd4dfe0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:45.534140 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:45.590294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-cf767634-7f77-4cce-bd7a-9308fd4dfe0f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:45.592081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cf767634-7f77-4cce-bd7a-9308fd4dfe0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:45.684477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 709/1422] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:45.833591 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb504fcc-f775-4825-96ba-54f91d03eb8c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:45.835113 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bb504fcc-f775-4825-96ba-54f91d03eb8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:45.942679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:45.979018 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb504fcc-f775-4825-96ba-54f91d03eb8c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:46.091358 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21553d-51f7-4973-b791-96f25d612c05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:46.091921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21553d-51f7-4973-b791-96f25d612c05 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:46.092288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21553d-51f7-4973-b791-96f25d612c05 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:46.092556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21553d-51f7-4973-b791-96f25d612c05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:46.094432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8e21553d-51f7-4973-b791-96f25d612c05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:46.303443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:46.323164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21553d-51f7-4973-b791-96f25d612c05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:46.330320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21553d-51f7-4973-b791-96f25d612c05 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:46.350151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 714/1423] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:47.597356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd549abe-0e63-4e21-b75c-113ce6d30034 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:47.597973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd549abe-0e63-4e21-b75c-113ce6d30034 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:47.597973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd549abe-0e63-4e21-b75c-113ce6d30034 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:47.599248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 710/1424] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:47.605843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9518006-6644-43ae-8848-5e7c7663877b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:47.606328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9518006-6644-43ae-8848-5e7c7663877b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:47.606433 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9518006-6644-43ae-8848-5e7c7663877b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:47.679551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d9518006-6644-43ae-8848-5e7c7663877b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:47.680753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d9518006-6644-43ae-8848-5e7c7663877b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:47.775076 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 715/1425] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:47.927191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd549abe-0e63-4e21-b75c-113ce6d30034 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:47.928691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cd549abe-0e63-4e21-b75c-113ce6d30034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:48.013258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:48.074456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd549abe-0e63-4e21-b75c-113ce6d30034 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:48.187823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69e46c-04a1-4dce-a046-522097593d09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:48.188457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69e46c-04a1-4dce-a046-522097593d09 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:48.188855 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69e46c-04a1-4dce-a046-522097593d09 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:48.189149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69e46c-04a1-4dce-a046-522097593d09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:48.190729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1c69e46c-04a1-4dce-a046-522097593d09 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:48.240864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:48.283335 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69e46c-04a1-4dce-a046-522097593d09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:48.284916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69e46c-04a1-4dce-a046-522097593d09 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:48.302769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 711/1426] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:49.372520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d667c9-c786-40cb-9ea4-d73d297a63a8 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:49.373062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d667c9-c786-40cb-9ea4-d73d297a63a8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:49.373204 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d667c9-c786-40cb-9ea4-d73d297a63a8 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:49.374565 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 716/1427] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:03:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:49.380709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3176a1b-1dc9-427f-8ba4-90af89d3d922 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:49.381160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3176a1b-1dc9-427f-8ba4-90af89d3d922 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:49.381322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3176a1b-1dc9-427f-8ba4-90af89d3d922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:49.456465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d3176a1b-1dc9-427f-8ba4-90af89d3d922 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:49.457702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d3176a1b-1dc9-427f-8ba4-90af89d3d922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:49.554530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 712/1428] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:49.705239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d667c9-c786-40cb-9ea4-d73d297a63a8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:49.706777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-18d667c9-c786-40cb-9ea4-d73d297a63a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:49.757439 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:49.852301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d667c9-c786-40cb-9ea4-d73d297a63a8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:49.963763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05e81e-cf89-49f2-b4e9-f73f4ead3d85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:49.964303 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05e81e-cf89-49f2-b4e9-f73f4ead3d85 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:49.964579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05e81e-cf89-49f2-b4e9-f73f4ead3d85 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:49.964847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05e81e-cf89-49f2-b4e9-f73f4ead3d85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:49.966511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0d05e81e-cf89-49f2-b4e9-f73f4ead3d85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:49.973575 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:50.058353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05e81e-cf89-49f2-b4e9-f73f4ead3d85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:50.060516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05e81e-cf89-49f2-b4e9-f73f4ead3d85 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:50.079786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 717/1429] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:51.132862 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234497d-5c83-477f-8a3c-d476d0257af4 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:51.133445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234497d-5c83-477f-8a3c-d476d0257af4 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:51.133445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234497d-5c83-477f-8a3c-d476d0257af4 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:51.134783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 713/1430] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:51.141262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e6cc7-118b-45f2-aa5a-a4832b8fc5f1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:51.141774 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e6cc7-118b-45f2-aa5a-a4832b8fc5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:51.141876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e6cc7-118b-45f2-aa5a-a4832b8fc5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:51.217609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6d9e6cc7-118b-45f2-aa5a-a4832b8fc5f1 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:51.218814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6d9e6cc7-118b-45f2-aa5a-a4832b8fc5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:51.285515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:51.316050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 718/1431] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:03:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:51.466682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234497d-5c83-477f-8a3c-d476d0257af4 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:51.468179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4234497d-5c83-477f-8a3c-d476d0257af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:51.489393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:51.618267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234497d-5c83-477f-8a3c-d476d0257af4 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:51.704375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:51.730645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311742f3-5858-4c11-8960-fe8577d0375a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:51.731238 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311742f3-5858-4c11-8960-fe8577d0375a service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:51.731518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311742f3-5858-4c11-8960-fe8577d0375a service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:51.731785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311742f3-5858-4c11-8960-fe8577d0375a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:51.733368 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-311742f3-5858-4c11-8960-fe8577d0375a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:51.820771 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311742f3-5858-4c11-8960-fe8577d0375a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:51.822443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311742f3-5858-4c11-8960-fe8577d0375a service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:51.841502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 714/1432] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:52.892281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4146d-57c4-42e8-8f95-af209d2ac75e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:52.892879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4146d-57c4-42e8-8f95-af209d2ac75e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:52.892879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d4146d-57c4-42e8-8f95-af209d2ac75e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:52.894212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 719/1433] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:52.900457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fd042-5372-4469-8820-d0f688f550e0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:52.900896 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fd042-5372-4469-8820-d0f688f550e0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:52.900993 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fd042-5372-4469-8820-d0f688f550e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:52.976169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-174fd042-5372-4469-8820-d0f688f550e0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:52.977402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-174fd042-5372-4469-8820-d0f688f550e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:53.018223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:53.074532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 715/1434] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:53.226548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4146d-57c4-42e8-8f95-af209d2ac75e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:53.227963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-69d4146d-57c4-42e8-8f95-af209d2ac75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:53.369895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d4146d-57c4-42e8-8f95-af209d2ac75e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:53.424136 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:53.482293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780a480-5300-4673-9f93-46d20972019b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:53.482908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a480-5300-4673-9f93-46d20972019b service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:53.483183 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a480-5300-4673-9f93-46d20972019b service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:53.483453 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780a480-5300-4673-9f93-46d20972019b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:53.485021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a780a480-5300-4673-9f93-46d20972019b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:53.574058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a480-5300-4673-9f93-46d20972019b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:53.576240 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780a480-5300-4673-9f93-46d20972019b service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:53.594380 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 720/1435] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:54.640598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c58382-8ab0-467e-8b19-feadc0422ed5 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:54.641193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c58382-8ab0-467e-8b19-feadc0422ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:54.641193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c58382-8ab0-467e-8b19-feadc0422ed5 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:54.642572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 716/1436] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:54.648630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2631f2-9363-43ec-8c37-d1bb7fdeafef None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:54.649051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2631f2-9363-43ec-8c37-d1bb7fdeafef None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:54.649232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2631f2-9363-43ec-8c37-d1bb7fdeafef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:54.724371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9a2631f2-9363-43ec-8c37-d1bb7fdeafef None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:54.725628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9a2631f2-9363-43ec-8c37-d1bb7fdeafef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:54.727611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:54.822028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 721/1437] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:54.996922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c58382-8ab0-467e-8b19-feadc0422ed5 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:54.998412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-20c58382-8ab0-467e-8b19-feadc0422ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:55.140346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c58382-8ab0-467e-8b19-feadc0422ed5 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:55.167146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:55.255197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275baf1a-1ab6-4bdc-a384-1de17cdbb7fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:55.255800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275baf1a-1ab6-4bdc-a384-1de17cdbb7fe service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:55.256076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275baf1a-1ab6-4bdc-a384-1de17cdbb7fe service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:55.256343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275baf1a-1ab6-4bdc-a384-1de17cdbb7fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:55.257967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-275baf1a-1ab6-4bdc-a384-1de17cdbb7fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:55.346318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275baf1a-1ab6-4bdc-a384-1de17cdbb7fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:55.347887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275baf1a-1ab6-4bdc-a384-1de17cdbb7fe service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:55.365886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 717/1438] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:03:57.421950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497cacc-c992-402f-ac08-72c34dfb4e9e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:57.422540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497cacc-c992-402f-ac08-72c34dfb4e9e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:57.422540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1497cacc-c992-402f-ac08-72c34dfb4e9e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:57.423855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 722/1439] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:03:57.429077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bcfba-d617-4df9-a412-ab5184a08b90 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:57.429497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bcfba-d617-4df9-a412-ab5184a08b90 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:57.429661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739bcfba-d617-4df9-a412-ab5184a08b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:57.471152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:57.495957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-739bcfba-d617-4df9-a412-ab5184a08b90 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:03:57.497847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-739bcfba-d617-4df9-a412-ab5184a08b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:57.594427 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 718/1440] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:03:57.880205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497cacc-c992-402f-ac08-72c34dfb4e9e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:03:57.881760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1497cacc-c992-402f-ac08-72c34dfb4e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:57.985291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1497cacc-c992-402f-ac08-72c34dfb4e9e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:03:58.017479 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:03:58.061142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf40328-aee4-43ec-80df-0b0f5855a199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:03:58.061748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf40328-aee4-43ec-80df-0b0f5855a199 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:03:58.062026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf40328-aee4-43ec-80df-0b0f5855a199 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:03:58.062294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf40328-aee4-43ec-80df-0b0f5855a199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:03:58.063888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7bf40328-aee4-43ec-80df-0b0f5855a199 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:03:58.171551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf40328-aee4-43ec-80df-0b0f5855a199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:03:58.172902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf40328-aee4-43ec-80df-0b0f5855a199 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:03:58.194347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 723/1441] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:03:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:00.571182 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77849502-d032-4435-8bf7-620ba7dabbb2 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:00.571753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77849502-d032-4435-8bf7-620ba7dabbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:00.571753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77849502-d032-4435-8bf7-620ba7dabbb2 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:00.573045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 719/1442] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:00.578771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1c618-33e3-49c7-b86d-d3f74456dc04 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:00.579201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1c618-33e3-49c7-b86d-d3f74456dc04 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:00.579309 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1c618-33e3-49c7-b86d-d3f74456dc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:00.643563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ada1c618-33e3-49c7-b86d-d3f74456dc04 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:00.644810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ada1c618-33e3-49c7-b86d-d3f74456dc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:00.658264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:00.743071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 724/1443] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:01.030299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77849502-d032-4435-8bf7-620ba7dabbb2 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:01.031803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-77849502-d032-4435-8bf7-620ba7dabbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:01.133420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77849502-d032-4435-8bf7-620ba7dabbb2 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:01.146587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:01.207636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0daebd4-4840-4973-b9bc-1e3e256efe1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:01.208237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daebd4-4840-4973-b9bc-1e3e256efe1d service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:01.208513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daebd4-4840-4973-b9bc-1e3e256efe1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:01.208782 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daebd4-4840-4973-b9bc-1e3e256efe1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:01.210389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e0daebd4-4840-4973-b9bc-1e3e256efe1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:01.320197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0daebd4-4840-4973-b9bc-1e3e256efe1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:01.321579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0daebd4-4840-4973-b9bc-1e3e256efe1d service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:01.340020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 720/1444] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:03.854517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451a40b-2a82-4582-bba8-e0aedbcfbc11 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:03.855630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451a40b-2a82-4582-bba8-e0aedbcfbc11 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:03.855630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451a40b-2a82-4582-bba8-e0aedbcfbc11 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:03.857226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 725/1445] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:04:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:03.862598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b9565-94fc-41cb-bae6-de6c4c5b3b6e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:03.863034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b9565-94fc-41cb-bae6-de6c4c5b3b6e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:03.863131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63b9565-94fc-41cb-bae6-de6c4c5b3b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:03.909994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:03.929043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f63b9565-94fc-41cb-bae6-de6c4c5b3b6e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:03.930264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f63b9565-94fc-41cb-bae6-de6c4c5b3b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:04.026019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 721/1446] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:04:04.311627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451a40b-2a82-4582-bba8-e0aedbcfbc11 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:04.313203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e451a40b-2a82-4582-bba8-e0aedbcfbc11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:04.415119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451a40b-2a82-4582-bba8-e0aedbcfbc11 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:04.455726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:04.489398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06c323-fd88-48c0-a1cd-b84f37bb61ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:04.490109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06c323-fd88-48c0-a1cd-b84f37bb61ef service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:04.490395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06c323-fd88-48c0-a1cd-b84f37bb61ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:04.490665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06c323-fd88-48c0-a1cd-b84f37bb61ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:04.492243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0a06c323-fd88-48c0-a1cd-b84f37bb61ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:04.598508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06c323-fd88-48c0-a1cd-b84f37bb61ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:04.599856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06c323-fd88-48c0-a1cd-b84f37bb61ef service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:04.617378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 726/1447] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:06.887270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24886a1a-5111-4c13-a89f-841bf2841b39 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:06.887845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24886a1a-5111-4c13-a89f-841bf2841b39 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:06.887845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24886a1a-5111-4c13-a89f-841bf2841b39 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:06.889175 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 722/1448] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:04:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:06.894428 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6f0b0-79dc-4b6a-b126-4f8834b0599b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:06.894859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6f0b0-79dc-4b6a-b126-4f8834b0599b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:06.894988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6f0b0-79dc-4b6a-b126-4f8834b0599b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:06.962026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-02b6f0b0-79dc-4b6a-b126-4f8834b0599b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:06.963265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-02b6f0b0-79dc-4b6a-b126-4f8834b0599b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:06.984606 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:07.060328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 727/1449] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:07.341834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24886a1a-5111-4c13-a89f-841bf2841b39 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:07.343353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-24886a1a-5111-4c13-a89f-841bf2841b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:07.444915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24886a1a-5111-4c13-a89f-841bf2841b39 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:07.464994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:07.519081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5edbce-e13a-4057-ad8c-54e4eb048e8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:07.519677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5edbce-e13a-4057-ad8c-54e4eb048e8b service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:07.519949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5edbce-e13a-4057-ad8c-54e4eb048e8b service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:07.520212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5edbce-e13a-4057-ad8c-54e4eb048e8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:07.521814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4f5edbce-e13a-4057-ad8c-54e4eb048e8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:07.629209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5edbce-e13a-4057-ad8c-54e4eb048e8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:07.630558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5edbce-e13a-4057-ad8c-54e4eb048e8b service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:07.648640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 723/1450] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:09.956927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fead805-2923-49da-82ea-552901843c9c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:09.957548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fead805-2923-49da-82ea-552901843c9c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:09.957548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fead805-2923-49da-82ea-552901843c9c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:09.958889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 728/1451] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:09.965013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19427-f11c-4d0a-adc3-6dcf0548a1a8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:09.965496 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19427-f11c-4d0a-adc3-6dcf0548a1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:09.965595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d19427-f11c-4d0a-adc3-6dcf0548a1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:10.028915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:10.032212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-04d19427-f11c-4d0a-adc3-6dcf0548a1a8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:10.034019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-04d19427-f11c-4d0a-adc3-6dcf0548a1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:10.129457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 724/1452] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:10.413578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fead805-2923-49da-82ea-552901843c9c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:10.415090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8fead805-2923-49da-82ea-552901843c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:10.519910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fead805-2923-49da-82ea-552901843c9c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:10.569777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:10.594628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f77fd2b-96d5-4fbe-af88-ea55405cdb45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:10.595235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77fd2b-96d5-4fbe-af88-ea55405cdb45 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:10.595510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77fd2b-96d5-4fbe-af88-ea55405cdb45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:10.595777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f77fd2b-96d5-4fbe-af88-ea55405cdb45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:10.597389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8f77fd2b-96d5-4fbe-af88-ea55405cdb45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:10.706900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77fd2b-96d5-4fbe-af88-ea55405cdb45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:10.708268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f77fd2b-96d5-4fbe-af88-ea55405cdb45 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:10.725857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 729/1453] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:12.928441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a4648-26e0-4b60-a463-b566651585da None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:12.928990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a4648-26e0-4b60-a463-b566651585da None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:12.928990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a4648-26e0-4b60-a463-b566651585da None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:12.930352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 725/1454] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:12.936373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227accd-aeb9-4a29-9882-35dc1cee9bee None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:12.936788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227accd-aeb9-4a29-9882-35dc1cee9bee None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:12.936891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227accd-aeb9-4a29-9882-35dc1cee9bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:13.001774 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4227accd-aeb9-4a29-9882-35dc1cee9bee None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:13.002993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4227accd-aeb9-4a29-9882-35dc1cee9bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:13.031262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:13.099050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 730/1455] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:04:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:04:13.387864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a4648-26e0-4b60-a463-b566651585da None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:13.389423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a62a4648-26e0-4b60-a463-b566651585da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:13.491463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a4648-26e0-4b60-a463-b566651585da None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:13.519817 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:13.564492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ef8c1-1157-4e62-8dc0-ea9c6157d7cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:13.565089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ef8c1-1157-4e62-8dc0-ea9c6157d7cb service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:13.565393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ef8c1-1157-4e62-8dc0-ea9c6157d7cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:13.565662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ef8c1-1157-4e62-8dc0-ea9c6157d7cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:13.567231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-372ef8c1-1157-4e62-8dc0-ea9c6157d7cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:13.674464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ef8c1-1157-4e62-8dc0-ea9c6157d7cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:13.675935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372ef8c1-1157-4e62-8dc0-ea9c6157d7cb service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:13.694429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 726/1456] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:15.912265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f31b8-0b29-4f81-ab96-0801ff04b63b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:15.912833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f31b8-0b29-4f81-ab96-0801ff04b63b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:15.912833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1f31b8-0b29-4f81-ab96-0801ff04b63b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:15.914284 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 731/1457] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:15.919674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea95d6-8094-4a05-ac1a-df82d8a538d6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:15.920065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea95d6-8094-4a05-ac1a-df82d8a538d6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:15.920223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea95d6-8094-4a05-ac1a-df82d8a538d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:15.984711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-37ea95d6-8094-4a05-ac1a-df82d8a538d6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:15.985928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-37ea95d6-8094-4a05-ac1a-df82d8a538d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:15.993954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:16.084890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 727/1458] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:16.363878 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f31b8-0b29-4f81-ab96-0801ff04b63b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:16.365462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ab1f31b8-0b29-4f81-ab96-0801ff04b63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:16.468709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1f31b8-0b29-4f81-ab96-0801ff04b63b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:16.523317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:16.540924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188d84f-2c16-400f-a796-f23da282e851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:16.542061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188d84f-2c16-400f-a796-f23da282e851 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:16.542427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188d84f-2c16-400f-a796-f23da282e851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:16.542698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188d84f-2c16-400f-a796-f23da282e851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:16.544253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0188d84f-2c16-400f-a796-f23da282e851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:16.649404 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188d84f-2c16-400f-a796-f23da282e851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:16.650738 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188d84f-2c16-400f-a796-f23da282e851 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:16.668842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 732/1459] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:18.877250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af68951-32e5-4dc8-8d78-4397183aff56 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:18.877860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af68951-32e5-4dc8-8d78-4397183aff56 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:18.877860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af68951-32e5-4dc8-8d78-4397183aff56 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:18.879211 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 728/1460] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:04:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:18.884522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea7a6c-9340-4db5-979d-779ad1b8a332 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:18.884984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea7a6c-9340-4db5-979d-779ad1b8a332 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:18.885096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea7a6c-9340-4db5-979d-779ad1b8a332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:18.949258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-38ea7a6c-9340-4db5-979d-779ad1b8a332 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:18.950441 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-38ea7a6c-9340-4db5-979d-779ad1b8a332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:18.987130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:19.045429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 733/1461] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:19.332280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af68951-32e5-4dc8-8d78-4397183aff56 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:19.333858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9af68951-32e5-4dc8-8d78-4397183aff56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:19.437525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af68951-32e5-4dc8-8d78-4397183aff56 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:19.475419 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:19.511272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ba0be-e814-45e3-8900-4a7e7408ae89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:19.511880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ba0be-e814-45e3-8900-4a7e7408ae89 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:19.512156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ba0be-e814-45e3-8900-4a7e7408ae89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:19.512424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ba0be-e814-45e3-8900-4a7e7408ae89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:19.514031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f63ba0be-e814-45e3-8900-4a7e7408ae89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:19.620745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ba0be-e814-45e3-8900-4a7e7408ae89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:19.622193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ba0be-e814-45e3-8900-4a7e7408ae89 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:19.640236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 729/1462] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:21.832256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8fcf5-28f3-4939-b54b-ec4bd541bff9 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:21.832843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8fcf5-28f3-4939-b54b-ec4bd541bff9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:21.832843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8fcf5-28f3-4939-b54b-ec4bd541bff9 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:21.834174 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 734/1463] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:21.839560 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0255f-645e-49d5-8cd5-23752ac498ca None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:21.840014 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0255f-645e-49d5-8cd5-23752ac498ca None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:21.840126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b0255f-645e-49d5-8cd5-23752ac498ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:21.905341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c6b0255f-645e-49d5-8cd5-23752ac498ca None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:21.906553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c6b0255f-645e-49d5-8cd5-23752ac498ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:21.924599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:22.003102 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 730/1464] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:04:22.284064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8fcf5-28f3-4939-b54b-ec4bd541bff9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:22.285688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-76f8fcf5-28f3-4939-b54b-ec4bd541bff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:22.386699 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8fcf5-28f3-4939-b54b-ec4bd541bff9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:22.454651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:22.461455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855eb9af-007c-4d51-883e-3693e20f6987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:22.462061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855eb9af-007c-4d51-883e-3693e20f6987 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:22.462340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855eb9af-007c-4d51-883e-3693e20f6987 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:22.462613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855eb9af-007c-4d51-883e-3693e20f6987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:22.464864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-855eb9af-007c-4d51-883e-3693e20f6987 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:22.571009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855eb9af-007c-4d51-883e-3693e20f6987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:22.573095 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855eb9af-007c-4d51-883e-3693e20f6987 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:22.591694 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 735/1465] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:24.746897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab453c1-cb96-4a83-bfb5-2166abe11ede None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:24.747534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab453c1-cb96-4a83-bfb5-2166abe11ede None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:24.747534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab453c1-cb96-4a83-bfb5-2166abe11ede None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:24.748781 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 731/1466] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:24.754444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66655f-7802-4e12-8f94-78ae6ad8a4cd None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:24.754885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66655f-7802-4e12-8f94-78ae6ad8a4cd None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:24.754985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa66655f-7802-4e12-8f94-78ae6ad8a4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:24.820290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fa66655f-7802-4e12-8f94-78ae6ad8a4cd None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:24.821534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fa66655f-7802-4e12-8f94-78ae6ad8a4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:24.868705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:24.918171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 736/1467] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:25.203125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab453c1-cb96-4a83-bfb5-2166abe11ede None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:25.204646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4ab453c1-cb96-4a83-bfb5-2166abe11ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:25.306307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab453c1-cb96-4a83-bfb5-2166abe11ede None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:25.354898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:25.380774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08eba8d-746a-4d22-8884-84dfaa284e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:25.381387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08eba8d-746a-4d22-8884-84dfaa284e2f service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:25.381669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08eba8d-746a-4d22-8884-84dfaa284e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:25.381939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08eba8d-746a-4d22-8884-84dfaa284e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:25.383519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f08eba8d-746a-4d22-8884-84dfaa284e2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:25.491030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08eba8d-746a-4d22-8884-84dfaa284e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:25.492457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08eba8d-746a-4d22-8884-84dfaa284e2f service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:25.510571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 732/1468] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:34.916343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=22 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:35.507813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=66 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:45.530228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7a4cc-de43-4e95-b873-e0444cc0024c None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:45.530678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7a4cc-de43-4e95-b873-e0444cc0024c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:45.530779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7a4cc-de43-4e95-b873-e0444cc0024c None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:45.532126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 737/1469] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:45.537480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031632b-b9cf-46db-a897-d702da72653f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:45.537912 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031632b-b9cf-46db-a897-d702da72653f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:45.538026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031632b-b9cf-46db-a897-d702da72653f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:45.602731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a031632b-b9cf-46db-a897-d702da72653f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:45.603920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a031632b-b9cf-46db-a897-d702da72653f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:45.698191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 733/1470] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:45.995972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2c48a-cc57-4c53-a99d-1662a7862a85 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:45.996287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2c48a-cc57-4c53-a99d-1662a7862a85 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:45.996568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b2c48a-cc57-4c53-a99d-1662a7862a85 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:45.997939 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 738/1471] 10.222.0.25 () {54 vars in 962 bytes} [Thu Sep 22 22:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:46.002936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cb92b-4357-4f73-91cc-36577f402dff None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:46.003223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cb92b-4357-4f73-91cc-36577f402dff None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:46.003498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cb92b-4357-4f73-91cc-36577f402dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:46.079546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-812cb92b-4357-4f73-91cc-36577f402dff None None] MFA Rules not processed for user `195565a7695b4d9b82e0ee76181fee2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:46.080748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-812cb92b-4357-4f73-91cc-36577f402dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:46.102032 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:46.162679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 734/1472] 10.222.0.25 () {58 vars in 1059 bytes} [Thu Sep 22 22:04:46 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:46.171360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2c48a-cc57-4c53-a99d-1662a7862a85 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:46.172855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-50b2c48a-cc57-4c53-a99d-1662a7862a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:46.296182 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b2c48a-cc57-4c53-a99d-1662a7862a85 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:46.380472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef422d40-ee52-4331-a0c3-5e4d62b18e96 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '195565a7695b4d9b82e0ee76181fee2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:46.381099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef422d40-ee52-4331-a0c3-5e4d62b18e96 service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:46.381425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef422d40-ee52-4331-a0c3-5e4d62b18e96 service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:46.381907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef422d40-ee52-4331-a0c3-5e4d62b18e96 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:46.384140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ef422d40-ee52-4331-a0c3-5e4d62b18e96 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:46.441660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:46.505195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef422d40-ee52-4331-a0c3-5e4d62b18e96 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:46.507124 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef422d40-ee52-4331-a0c3-5e4d62b18e96 service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:46.532079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 739/1473] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:48.936869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e9598-8321-4df9-9a0c-4d5d12846996 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:48.937510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e9598-8321-4df9-9a0c-4d5d12846996 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:48.937510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e9598-8321-4df9-9a0c-4d5d12846996 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:48.938777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 735/1474] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:48.944135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0d2c6-639b-4332-a496-ab65381b16b7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:48.944597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0d2c6-639b-4332-a496-ab65381b16b7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:48.944709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0d2c6-639b-4332-a496-ab65381b16b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:49.011666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5ea0d2c6-639b-4332-a496-ab65381b16b7 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:49.012882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5ea0d2c6-639b-4332-a496-ab65381b16b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:49.110315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 740/1475] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:49.383587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e9598-8321-4df9-9a0c-4d5d12846996 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:49.385359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-101e9598-8321-4df9-9a0c-4d5d12846996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:49.509505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e9598-8321-4df9-9a0c-4d5d12846996 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:49.562005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:49.604996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c99f97c-7bec-40c2-b8f7-d62ababc0bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:49.605614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f97c-7bec-40c2-b8f7-d62ababc0bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:49.605889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f97c-7bec-40c2-b8f7-d62ababc0bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:49.606157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c99f97c-7bec-40c2-b8f7-d62ababc0bf6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:49.606781 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99f97c-7bec-40c2-b8f7-d62ababc0bf6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:49.608488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-7c99f97c-7bec-40c2-b8f7-d62ababc0bf6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:04:49.608488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 22 22:04:49.608488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: Sep 22 22:04:49.609642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c99f97c-7bec-40c2-b8f7-d62ababc0bf6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:49.628711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 736/1476] 10.222.0.25 () {56 vars in 1108 bytes} [Thu Sep 22 22:04:49 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:04:51.804066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff7a6f-4615-41ad-8b14-f0f788a1c757 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:51.804599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff7a6f-4615-41ad-8b14-f0f788a1c757 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:51.804599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff7a6f-4615-41ad-8b14-f0f788a1c757 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:51.805906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 741/1477] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:51.811233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e234f1e-f9b0-4ea5-bf98-369b49f8afab None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:51.811666 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e234f1e-f9b0-4ea5-bf98-369b49f8afab None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:51.811786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e234f1e-f9b0-4ea5-bf98-369b49f8afab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:51.878150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6e234f1e-f9b0-4ea5-bf98-369b49f8afab None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:51.879354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6e234f1e-f9b0-4ea5-bf98-369b49f8afab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:51.975421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 737/1478] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:04:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:52.268185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff7a6f-4615-41ad-8b14-f0f788a1c757 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:52.269722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-feff7a6f-4615-41ad-8b14-f0f788a1c757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:52.279680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:52.413765 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feff7a6f-4615-41ad-8b14-f0f788a1c757 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:52.493869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:52.525701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9c415c-644f-48b2-9638-943a632f80ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:52.526306 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c415c-644f-48b2-9638-943a632f80ef service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:52.526579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c415c-644f-48b2-9638-943a632f80ef service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:52.526842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c415c-644f-48b2-9638-943a632f80ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:52.528448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8e9c415c-644f-48b2-9638-943a632f80ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:52.616802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c415c-644f-48b2-9638-943a632f80ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:52.619137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9c415c-644f-48b2-9638-943a632f80ef service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:52.636697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 742/1479] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:54.960621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b779b-e880-41ba-9358-99550ae62726 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:54.961937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b779b-e880-41ba-9358-99550ae62726 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:54.961937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21b779b-e880-41ba-9358-99550ae62726 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:54.963174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 738/1480] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:04:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:54.968529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69e1c3-93fb-4529-8371-321b94eb38ce None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:54.968984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69e1c3-93fb-4529-8371-321b94eb38ce None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:54.969097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69e1c3-93fb-4529-8371-321b94eb38ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:55.033241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4a69e1c3-93fb-4529-8371-321b94eb38ce None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:55.034435 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4a69e1c3-93fb-4529-8371-321b94eb38ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:55.080917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:55.130852 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 743/1481] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:04:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:55.416680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b779b-e880-41ba-9358-99550ae62726 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:55.418233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e21b779b-e880-41ba-9358-99550ae62726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:55.441849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:55.523356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21b779b-e880-41ba-9358-99550ae62726 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:55.599791 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ac9b20-d048-42ec-9526-711fc65e4612 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:55.600395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac9b20-d048-42ec-9526-711fc65e4612 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:55.600674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac9b20-d048-42ec-9526-711fc65e4612 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:55.600947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ac9b20-d048-42ec-9526-711fc65e4612 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:55.603209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-95ac9b20-d048-42ec-9526-711fc65e4612 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:55.676651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:55.712026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac9b20-d048-42ec-9526-711fc65e4612 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:55.713490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ac9b20-d048-42ec-9526-711fc65e4612 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:55.731498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 739/1482] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:04:58.012522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a706140-eca5-43fb-9140-e3de84c2567e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:58.013070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a706140-eca5-43fb-9140-e3de84c2567e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:58.013070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a706140-eca5-43fb-9140-e3de84c2567e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:58.014432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 744/1483] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:04:58.019546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d516f-b598-48dd-bec6-a4092c4f2490 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:58.019979 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d516f-b598-48dd-bec6-a4092c4f2490 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:58.020084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d516f-b598-48dd-bec6-a4092c4f2490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:58.093440 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b88d516f-b598-48dd-bec6-a4092c4f2490 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:04:58.094684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b88d516f-b598-48dd-bec6-a4092c4f2490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:58.192166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 740/1484] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:04:58.481534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a706140-eca5-43fb-9140-e3de84c2567e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:04:58.483048 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8a706140-eca5-43fb-9140-e3de84c2567e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:58.584656 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a706140-eca5-43fb-9140-e3de84c2567e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:04:58.655992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:04:58.657557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612841b5-d5cc-4252-a436-e1584de39260 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:04:58.658223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612841b5-d5cc-4252-a436-e1584de39260 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:04:58.658538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612841b5-d5cc-4252-a436-e1584de39260 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:04:58.658996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612841b5-d5cc-4252-a436-e1584de39260 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:04:58.660903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-612841b5-d5cc-4252-a436-e1584de39260 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:04:58.768922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612841b5-d5cc-4252-a436-e1584de39260 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:04:58.770373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612841b5-d5cc-4252-a436-e1584de39260 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:04:58.789374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 745/1485] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:05:01.062821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe13aa-07fc-4ff0-a42f-ff2d61017c44 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:05:01.063226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe13aa-07fc-4ff0-a42f-ff2d61017c44 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:05:01.063369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe13aa-07fc-4ff0-a42f-ff2d61017c44 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:05:01.064715 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 741/1486] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:05:01.070322 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2503f-a06b-4d3f-bef3-c7141c71b1b4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:05:01.071003 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2503f-a06b-4d3f-bef3-c7141c71b1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:05:01.071119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b2503f-a06b-4d3f-bef3-c7141c71b1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:05:01.135491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-44b2503f-a06b-4d3f-bef3-c7141c71b1b4 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:05:01.136684 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-44b2503f-a06b-4d3f-bef3-c7141c71b1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:05:01.189298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:05:01.233994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 746/1487] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:05:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:05:01.527281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe13aa-07fc-4ff0-a42f-ff2d61017c44 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:05:01.528818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-53fe13aa-07fc-4ff0-a42f-ff2d61017c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:05:01.566339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:05:01.633264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe13aa-07fc-4ff0-a42f-ff2d61017c44 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:05:01.705753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780ea99-be84-4161-ba78-9e6830de5302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:05:01.706342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780ea99-be84-4161-ba78-9e6830de5302 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:05:01.706616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780ea99-be84-4161-ba78-9e6830de5302 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:05:01.706890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780ea99-be84-4161-ba78-9e6830de5302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:05:01.708437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4780ea99-be84-4161-ba78-9e6830de5302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:05:01.788219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:05:01.818688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4780ea99-be84-4161-ba78-9e6830de5302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:05:01.821068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4780ea99-be84-4161-ba78-9e6830de5302 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:05:01.842663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 742/1488] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:05:04.177806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07a26c-c416-4d91-9349-a3bf5b61cf94 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:05:04.178372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07a26c-c416-4d91-9349-a3bf5b61cf94 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:05:04.178676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd07a26c-c416-4d91-9349-a3bf5b61cf94 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:05:04.180021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 747/1489] 10.222.0.25 () {54 vars in 927 bytes} [Thu Sep 22 22:05:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:05:04.185276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e367b0-acbe-4279-9593-94cd5148e106 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:05:04.185735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e367b0-acbe-4279-9593-94cd5148e106 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:05:04.185844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e367b0-acbe-4279-9593-94cd5148e106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:05:04.251255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-32e367b0-acbe-4279-9593-94cd5148e106 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:05:04.252459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-32e367b0-acbe-4279-9593-94cd5148e106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:05:04.347739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 743/1490] 10.222.0.25 () {58 vars in 1024 bytes} [Thu Sep 22 22:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:05:04.666079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07a26c-c416-4d91-9349-a3bf5b61cf94 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:05:04.667689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fd07a26c-c416-4d91-9349-a3bf5b61cf94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:05:04.768455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd07a26c-c416-4d91-9349-a3bf5b61cf94 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:05:04.841467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc62f7c-bea7-4a72-b62e-444fc86f4ad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:05:04.842068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc62f7c-bea7-4a72-b62e-444fc86f4ad4 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:05:04.842345 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc62f7c-bea7-4a72-b62e-444fc86f4ad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:05:04.842615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc62f7c-bea7-4a72-b62e-444fc86f4ad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:05:04.844199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4bc62f7c-bea7-4a72-b62e-444fc86f4ad4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:05:04.852707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:05:04.954426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc62f7c-bea7-4a72-b62e-444fc86f4ad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:05:04.955769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc62f7c-bea7-4a72-b62e-444fc86f4ad4 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:05:04.973689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 748/1491] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:05:14.346260 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=82 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:05:14.971423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=50 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:39.136259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b412c7-da36-4142-9f94-c939ccc695fd None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:39.136848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b412c7-da36-4142-9f94-c939ccc695fd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:39.136848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b412c7-da36-4142-9f94-c939ccc695fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:39.212093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b4b412c7-da36-4142-9f94-c939ccc695fd None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:06:39.213324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b4b412c7-da36-4142-9f94-c939ccc695fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:39.310796 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 744/1492] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:06:39.319452 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e7ee3-96a8-490f-8a70-d51812c9a7c0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:39.319944 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e7ee3-96a8-490f-8a70-d51812c9a7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:39.320042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e7ee3-96a8-490f-8a70-d51812c9a7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:39.395560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b97e7ee3-96a8-490f-8a70-d51812c9a7c0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:06:39.396769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b97e7ee3-96a8-490f-8a70-d51812c9a7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:39.492449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 749/1493] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:06:39.497128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b412c7-da36-4142-9f94-c939ccc695fd None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:06:39.498333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b4b412c7-da36-4142-9f94-c939ccc695fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:39.509908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b412c7-da36-4142-9f94-c939ccc695fd None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:06:39.571164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:39.606704 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf14666-7af2-489b-817f-ab278fbc53bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:06:39.607305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf14666-7af2-489b-817f-ab278fbc53bd admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:39.607586 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf14666-7af2-489b-817f-ab278fbc53bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:39.607866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf14666-7af2-489b-817f-ab278fbc53bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:39.608498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf14666-7af2-489b-817f-ab278fbc53bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:06:39.608802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf14666-7af2-489b-817f-ab278fbc53bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:06:39.609853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-7cf14666-7af2-489b-817f-ab278fbc53bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:06:39.609853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 22 22:06:39.609853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: Sep 22 22:06:39.610730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf14666-7af2-489b-817f-ab278fbc53bd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:06:39.621946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 745/1494] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:06:39.630689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e7ee3-96a8-490f-8a70-d51812c9a7c0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:06:39.639748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e7ee3-96a8-490f-8a70-d51812c9a7c0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:06:39.701632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:39.735551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223d1d0-caaa-4b6a-99b9-d76ba6f2ebd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:06:39.736145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223d1d0-caaa-4b6a-99b9-d76ba6f2ebd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:39.736422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223d1d0-caaa-4b6a-99b9-d76ba6f2ebd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:39.736695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223d1d0-caaa-4b6a-99b9-d76ba6f2ebd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:39.737541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223d1d0-caaa-4b6a-99b9-d76ba6f2ebd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:06:39.739536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-9223d1d0-caaa-4b6a-99b9-d76ba6f2ebd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:06:39.739536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 22 22:06:39.739536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: Sep 22 22:06:39.740731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223d1d0-caaa-4b6a-99b9-d76ba6f2ebd1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:06:39.796754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 750/1495] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:06:39 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:06:39.801082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf14666-7af2-489b-817f-ab278fbc53bd admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:06:39.802646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7cf14666-7af2-489b-817f-ab278fbc53bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:39.837231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf14666-7af2-489b-817f-ab278fbc53bd admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:06:39.930560 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a3102-d56e-452c-b3c9-1ffa579dadde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:06:39.931155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a3102-d56e-452c-b3c9-1ffa579dadde admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:39.931437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a3102-d56e-452c-b3c9-1ffa579dadde admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:39.931707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413a3102-d56e-452c-b3c9-1ffa579dadde admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:39.932327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a3102-d56e-452c-b3c9-1ffa579dadde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:06:39.933359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-413a3102-d56e-452c-b3c9-1ffa579dadde admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:06:39.933359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 22 22:06:39.933359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: Sep 22 22:06:39.934557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413a3102-d56e-452c-b3c9-1ffa579dadde admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:06:39.976935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 746/1496] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:06:39 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:06:39.981093 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223d1d0-caaa-4b6a-99b9-d76ba6f2ebd1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:06:39.982632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9223d1d0-caaa-4b6a-99b9-d76ba6f2ebd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:40.043143 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223d1d0-caaa-4b6a-99b9-d76ba6f2ebd1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:06:40.129056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:40.140818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d878eb-0d1a-47ad-ad85-e698861ccadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:06:40.141470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d878eb-0d1a-47ad-ad85-e698861ccadd admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:40.141744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d878eb-0d1a-47ad-ad85-e698861ccadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:40.142038 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d878eb-0d1a-47ad-ad85-e698861ccadd admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:40.142687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d878eb-0d1a-47ad-ad85-e698861ccadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:06:40.143799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-33d878eb-0d1a-47ad-ad85-e698861ccadd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:06:40.143799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 22 22:06:40.143799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: Sep 22 22:06:40.144672 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d878eb-0d1a-47ad-ad85-e698861ccadd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:06:40.156763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 751/1497] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:06:39 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:06:40.160880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a3102-d56e-452c-b3c9-1ffa579dadde admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:06:40.162408 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-413a3102-d56e-452c-b3c9-1ffa579dadde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:40.173611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413a3102-d56e-452c-b3c9-1ffa579dadde admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:06:40.199852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:40.270735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af862a24-42ac-454b-afd1-7b2db9318a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:06:40.271342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af862a24-42ac-454b-afd1-7b2db9318a94 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:40.271624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af862a24-42ac-454b-afd1-7b2db9318a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:40.271894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af862a24-42ac-454b-afd1-7b2db9318a94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:40.272532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af862a24-42ac-454b-afd1-7b2db9318a94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:06:40.273593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-af862a24-42ac-454b-afd1-7b2db9318a94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:06:40.273593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 22 22:06:40.273593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: Sep 22 22:06:40.274464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af862a24-42ac-454b-afd1-7b2db9318a94 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:06:40.286997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 747/1498] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:06:40.291109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d878eb-0d1a-47ad-ad85-e698861ccadd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:06:40.292602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-33d878eb-0d1a-47ad-ad85-e698861ccadd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:40.332806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d878eb-0d1a-47ad-ad85-e698861ccadd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:06:40.426424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e1e72c-0415-4317-96b8-4ac38bdf0b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:06:40.427003 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1e72c-0415-4317-96b8-4ac38bdf0b44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:40.427286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1e72c-0415-4317-96b8-4ac38bdf0b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:40.427555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e1e72c-0415-4317-96b8-4ac38bdf0b44 admin admin] PATH_INFO: `/v3/projects/040de28abcc2440f881d6b971977cf70/users/3aa351fe3e904c4f81bc3d15f32a79b5/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:40.449795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1e72c-0415-4317-96b8-4ac38bdf0b44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=040de28abcc2440f881d6b971977cf70, user_id=3aa351fe3e904c4f81bc3d15f32a79b5, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:06:40.450819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-03e1e72c-0415-4317-96b8-4ac38bdf0b44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:06:40.450819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 22 22:06:40.450819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: Sep 22 22:06:40.452736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e1e72c-0415-4317-96b8-4ac38bdf0b44 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:06:40.476826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 752/1499] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:06:40 2022] PUT /identity/v3/projects/040de28abcc2440f881d6b971977cf70/users/3aa351fe3e904c4f81bc3d15f32a79b5/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:06:40.489568 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dabe4-bb2f-46ef-8c16-3daa649c4ad4 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:40.489852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dabe4-bb2f-46ef-8c16-3daa649c4ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:40.490127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055dabe4-bb2f-46ef-8c16-3daa649c4ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:40.546566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-055dabe4-bb2f-46ef-8c16-3daa649c4ad4 None None] MFA Rules not processed for user `3aa351fe3e904c4f81bc3d15f32a79b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:06:40.547788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-055dabe4-bb2f-46ef-8c16-3daa649c4ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:40.622881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 748/1500] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:06:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:06:40.635907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36047d74-2b6e-4ba4-8693-1e080a59299d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:40.636189 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36047d74-2b6e-4ba4-8693-1e080a59299d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:40.636463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36047d74-2b6e-4ba4-8693-1e080a59299d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:40.637833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 753/1501] 10.222.0.25 () {54 vars in 963 bytes} [Thu Sep 22 22:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:06:40.642588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619cf15-f0c9-4417-a8fd-88353d222177 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:40.642875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619cf15-f0c9-4417-a8fd-88353d222177 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:40.643143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f619cf15-f0c9-4417-a8fd-88353d222177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:40.674384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:40.718621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f619cf15-f0c9-4417-a8fd-88353d222177 None None] MFA Rules not processed for user `fcfdbb6b3dd04478ae99681cc649fdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:06:40.719739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f619cf15-f0c9-4417-a8fd-88353d222177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:40.795355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 749/1502] 10.222.0.25 () {58 vars in 1060 bytes} [Thu Sep 22 22:06:40 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:06:40.803880 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36047d74-2b6e-4ba4-8693-1e080a59299d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:06:40.805563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-36047d74-2b6e-4ba4-8693-1e080a59299d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:40.858354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:40.913261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36047d74-2b6e-4ba4-8693-1e080a59299d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:06:40.985858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351303c-8567-4ca3-8d46-b98e35e07113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:06:40.986386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351303c-8567-4ca3-8d46-b98e35e07113 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:40.986658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351303c-8567-4ca3-8d46-b98e35e07113 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:40.986922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2351303c-8567-4ca3-8d46-b98e35e07113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:40.988571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2351303c-8567-4ca3-8d46-b98e35e07113 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:41.090967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351303c-8567-4ca3-8d46-b98e35e07113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:06:41.092383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2351303c-8567-4ca3-8d46-b98e35e07113 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:06:41.095841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:41.111068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 754/1503] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:06:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:06:41.157022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b11127-998a-4f03-b1b0-8ce5163bd339 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:41.157349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b11127-998a-4f03-b1b0-8ce5163bd339 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:41.157630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b11127-998a-4f03-b1b0-8ce5163bd339 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:41.158890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 750/1504] 10.222.0.25 () {52 vars in 830 bytes} [Thu Sep 22 22:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:06:41.163133 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351303c-8567-4ca3-8d46-b98e35e07113 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:06:41.164342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2351303c-8567-4ca3-8d46-b98e35e07113 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:41.263036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2351303c-8567-4ca3-8d46-b98e35e07113 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:06:41.355242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:06:41.355862 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:41.356124 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:41.356393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] PATH_INFO: `/v3/users/3aa351fe3e904c4f81bc3d15f32a79b5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:41.369586 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa351fe3e904c4f81bc3d15f32a79b5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:06:41.370600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:06:41.370600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 22 22:06:41.370600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: Sep 22 22:06:41.371798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:06:41.381133 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:41.437296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa351fe3e904c4f81bc3d15f32a79b5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:06:41.437825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa351fe3e904c4f81bc3d15f32a79b5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:06:41.438243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa351fe3e904c4f81bc3d15f32a79b5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:06:41.461356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa351fe3e904c4f81bc3d15f32a79b5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:06:41.475650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fe96e07e-ca54-4af1-b7f2-2b2b2df04477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa351fe3e904c4f81bc3d15f32a79b5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:06:41.490404 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 755/1505] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:06:41 2022] DELETE /identity/v3/users/3aa351fe3e904c4f81bc3d15f32a79b5 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:06:41.494777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793ff25-be83-4cd1-b6e5-aaf6d355e4fc None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:41.495268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793ff25-be83-4cd1-b6e5-aaf6d355e4fc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:41.495375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793ff25-be83-4cd1-b6e5-aaf6d355e4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:41.552568 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7793ff25-be83-4cd1-b6e5-aaf6d355e4fc None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:06:41.553741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7793ff25-be83-4cd1-b6e5-aaf6d355e4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:41.628454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:41.640683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 751/1506] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:06:41.667616 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af1162-fce9-4150-a44e-84d0957cf7d7 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:41.667901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af1162-fce9-4150-a44e-84d0957cf7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:41.668177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af1162-fce9-4150-a44e-84d0957cf7d7 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:41.669564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 756/1507] 10.222.0.25 () {54 vars in 964 bytes} [Thu Sep 22 22:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:06:41.675764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb08a77-23e9-47df-b79f-cb2be0ca7a69 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:41.676077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb08a77-23e9-47df-b79f-cb2be0ca7a69 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:41.676347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb08a77-23e9-47df-b79f-cb2be0ca7a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:41.751546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fcb08a77-23e9-47df-b79f-cb2be0ca7a69 None None] MFA Rules not processed for user `18c956fa035f40fa88fbf662541121ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:06:41.752787 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fcb08a77-23e9-47df-b79f-cb2be0ca7a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:41.827293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 752/1508] 10.222.0.25 () {58 vars in 1061 bytes} [Thu Sep 22 22:06:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:06:41.836798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af1162-fce9-4150-a44e-84d0957cf7d7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:06:41.838362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-12af1162-fce9-4150-a44e-84d0957cf7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:41.941976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af1162-fce9-4150-a44e-84d0957cf7d7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:06:41.991891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:42.016092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143460b-2848-4b3d-b6cd-b1433d465aa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:06:42.016686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143460b-2848-4b3d-b6cd-b1433d465aa1 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:42.016960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143460b-2848-4b3d-b6cd-b1433d465aa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:42.017262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143460b-2848-4b3d-b6cd-b1433d465aa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:42.019472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2143460b-2848-4b3d-b6cd-b1433d465aa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:42.128222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143460b-2848-4b3d-b6cd-b1433d465aa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:06:42.129608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143460b-2848-4b3d-b6cd-b1433d465aa1 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:06:42.147249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 757/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:06:42.885501 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb08a77-23e9-47df-b79f-cb2be0ca7a69 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:06:42.886767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fcb08a77-23e9-47df-b79f-cb2be0ca7a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:06:42.968565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:43.003866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb08a77-23e9-47df-b79f-cb2be0ca7a69 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:06:43.098226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:06:43.098852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:06:43.099127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:06:43.099392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] PATH_INFO: `/v3/projects/040de28abcc2440f881d6b971977cf70` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:06:43.109055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] RBAC: Authorizing `identity:delete_project(project_id=040de28abcc2440f881d6b971977cf70)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:06:43.110096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:06:43.110096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 22 22:06:43.110096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: Sep 22 22:06:43.111579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:06:43.170911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '040de28abcc2440f881d6b971977cf70', 'request_id': 'req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:06:43.181373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '040de28abcc2440f881d6b971977cf70', 'request_id': 'req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:06:43.195503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '040de28abcc2440f881d6b971977cf70', 'request_id': 'req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:06:43.219016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] The token cache is being invalidate because project 040de28abcc2440f881d6b971977cf70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:06:43.219304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:06:43.219731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8b39c6e9-0c0a-44a9-be7d-2ec772b39506 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:06:43.221124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 753/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:06:42 2022] DELETE /identity/v3/projects/040de28abcc2440f881d6b971977cf70 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:06:52.144928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=64 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:06:53.217023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:08:10.386289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161bee8-07f3-4110-9192-2ff8a2baffc2 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:10.386866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161bee8-07f3-4110-9192-2ff8a2baffc2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:10.386866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7161bee8-07f3-4110-9192-2ff8a2baffc2 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:10.388105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:08:10.393712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad0777-7b45-40f8-82f7-3a1932fa2d63 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:10.394168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad0777-7b45-40f8-82f7-3a1932fa2d63 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:10.394282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad0777-7b45-40f8-82f7-3a1932fa2d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:10.473224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-efad0777-7b45-40f8-82f7-3a1932fa2d63 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:08:10.474428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-efad0777-7b45-40f8-82f7-3a1932fa2d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:10.576992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:08:10.735771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161bee8-07f3-4110-9192-2ff8a2baffc2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:08:10.737900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7161bee8-07f3-4110-9192-2ff8a2baffc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:10.856589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7161bee8-07f3-4110-9192-2ff8a2baffc2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:08:10.861993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52bc858-aa93-4d2e-89a3-160339e54d11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:08:10.862589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bc858-aa93-4d2e-89a3-160339e54d11 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:10.862864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bc858-aa93-4d2e-89a3-160339e54d11 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:10.863133 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52bc858-aa93-4d2e-89a3-160339e54d11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:10.871046 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52bc858-aa93-4d2e-89a3-160339e54d11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:08:10.872604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52bc858-aa93-4d2e-89a3-160339e54d11 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:08:10.881858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 759/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:08:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:08:11.833229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0c6c3-40f5-4472-9096-22a111f84e2f None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:11.833755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0c6c3-40f5-4472-9096-22a111f84e2f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:11.833837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0c6c3-40f5-4472-9096-22a111f84e2f None None] PATH_INFO: `/v3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:11.835247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 755/1514] 10.222.0.57 () {54 vars in 925 bytes} [Thu Sep 22 22:08:11 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:08:11.842382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfde36-0369-4717-ba4e-6148b2f00a35 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:11.842809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfde36-0369-4717-ba4e-6148b2f00a35 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:11.843083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edfde36-0369-4717-ba4e-6148b2f00a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:11.906465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2edfde36-0369-4717-ba4e-6148b2f00a35 None None] MFA Rules not processed for user `43d6ba7b68884509a4e2859ceb72ef0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:08:11.908195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2edfde36-0369-4717-ba4e-6148b2f00a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:11.969328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bf62b-cf83-4478-a3c8-3fca0ba48e5b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:11.969608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bf62b-cf83-4478-a3c8-3fca0ba48e5b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:11.970372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658bf62b-cf83-4478-a3c8-3fca0ba48e5b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:11.971975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 756/1515] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:08:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 22:08:11.976856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:08:11.978103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99978e4-2113-4a4c-af71-969f558e9191 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:11.978332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99978e4-2113-4a4c-af71-969f558e9191 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:11.978595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99978e4-2113-4a4c-af71-969f558e9191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:11.982694 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 760/1516] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Sep 22 22:08:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:08:11.999627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b99978e4-2113-4a4c-af71-969f558e9191 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:08:12.000820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b99978e4-2113-4a4c-af71-969f558e9191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:12.008597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:08:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:08:12.161543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfde36-0369-4717-ba4e-6148b2f00a35 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:08:12.173368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edfde36-0369-4717-ba4e-6148b2f00a35 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:08:12.178760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6bbea-b5ba-4d1a-b78c-bfa676eb61f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:08:12.179354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6bbea-b5ba-4d1a-b78c-bfa676eb61f1 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:12.179630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6bbea-b5ba-4d1a-b78c-bfa676eb61f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:12.179900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6bbea-b5ba-4d1a-b78c-bfa676eb61f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:12.188646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6bbea-b5ba-4d1a-b78c-bfa676eb61f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:08:12.190233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6bbea-b5ba-4d1a-b78c-bfa676eb61f1 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:08:12.199364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 761/1518] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:08:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:08:20.649084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7098-d544-41b0-b29f-129b7e23b61b None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:20.649861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7098-d544-41b0-b29f-129b7e23b61b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:20.649861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc7098-d544-41b0-b29f-129b7e23b61b None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:20.651082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 758/1519] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 22 22:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:08:20.656055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d3491-22b5-4ec2-a0be-7a5f095e7a62 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:20.656532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d3491-22b5-4ec2-a0be-7a5f095e7a62 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:20.656638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d3491-22b5-4ec2-a0be-7a5f095e7a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:20.676477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fa0d3491-22b5-4ec2-a0be-7a5f095e7a62 None None] MFA Rules not processed for user `43d6ba7b68884509a4e2859ceb72ef0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:08:20.677700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fa0d3491-22b5-4ec2-a0be-7a5f095e7a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:20.691592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 762/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 22 22:08:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:08:20.699194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc7098-d544-41b0-b29f-129b7e23b61b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:08:20.712440 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc7098-d544-41b0-b29f-129b7e23b61b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:08:20.717410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032b663d-d3ff-4ae9-8c37-49f08260033d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:08:20.717989 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b663d-d3ff-4ae9-8c37-49f08260033d service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:20.718267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b663d-d3ff-4ae9-8c37-49f08260033d service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:20.718535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b663d-d3ff-4ae9-8c37-49f08260033d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:20.726737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b663d-d3ff-4ae9-8c37-49f08260033d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:08:20.728306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032b663d-d3ff-4ae9-8c37-49f08260033d service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:08:20.738442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 759/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:08:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:08:23.677235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa56adf-88c8-4c16-9e0e-68b444554949 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:23.677740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa56adf-88c8-4c16-9e0e-68b444554949 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:23.677845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa56adf-88c8-4c16-9e0e-68b444554949 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:23.679136 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:08:23.685361 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de11c-5861-485b-aa9d-38ab70b20369 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:23.685831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de11c-5861-485b-aa9d-38ab70b20369 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:23.685947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de11c-5861-485b-aa9d-38ab70b20369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:23.706337 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c32de11c-5861-485b-aa9d-38ab70b20369 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:08:23.707522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c32de11c-5861-485b-aa9d-38ab70b20369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:23.715305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:08:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:08:23.869909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa56adf-88c8-4c16-9e0e-68b444554949 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:08:23.884371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa56adf-88c8-4c16-9e0e-68b444554949 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:08:23.889628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f393c-3ad9-481d-82c1-c657b540b677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:08:23.890217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f393c-3ad9-481d-82c1-c657b540b677 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:23.890492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f393c-3ad9-481d-82c1-c657b540b677 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:23.890761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f393c-3ad9-481d-82c1-c657b540b677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:23.898588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f393c-3ad9-481d-82c1-c657b540b677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:08:23.900151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f393c-3ad9-481d-82c1-c657b540b677 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:08:23.909791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 764/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:08:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:08:24.987547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435d3a1-ceb0-4a3d-b9c6-303e804556ff None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:24.988117 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435d3a1-ceb0-4a3d-b9c6-303e804556ff None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:24.988117 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435d3a1-ceb0-4a3d-b9c6-303e804556ff None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:24.989502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 22 22:08:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:08:24.995474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46182d-4dd7-4b99-b95a-776889267740 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:24.995918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46182d-4dd7-4b99-b95a-776889267740 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:24.996027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46182d-4dd7-4b99-b95a-776889267740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:25.015986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0d46182d-4dd7-4b99-b95a-776889267740 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:08:25.017798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0d46182d-4dd7-4b99-b95a-776889267740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:25.025506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 22 22:08:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:08:25.179136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435d3a1-ceb0-4a3d-b9c6-303e804556ff None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:08:25.193694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435d3a1-ceb0-4a3d-b9c6-303e804556ff None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:08:25.199106 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7b388-4da0-49a2-b673-3f587bff4eb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:08:25.199690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7b388-4da0-49a2-b673-3f587bff4eb1 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:25.199966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7b388-4da0-49a2-b673-3f587bff4eb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:25.200237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7b388-4da0-49a2-b673-3f587bff4eb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:25.208760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7b388-4da0-49a2-b673-3f587bff4eb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:08:25.210381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7b388-4da0-49a2-b673-3f587bff4eb1 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:08:25.220343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 762/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:08:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:08:29.185512 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1768c01-0fd1-4e3f-b9f3-339e18a9522d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:29.186078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1768c01-0fd1-4e3f-b9f3-339e18a9522d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:29.186078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1768c01-0fd1-4e3f-b9f3-339e18a9522d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:29.187673 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:08:29.193600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9a87c-eb0e-4efd-b0d8-a4101b3a4364 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:29.193999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9a87c-eb0e-4efd-b0d8-a4101b3a4364 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:29.194148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9a87c-eb0e-4efd-b0d8-a4101b3a4364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:29.214432 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ebe9a87c-eb0e-4efd-b0d8-a4101b3a4364 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:08:29.215624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ebe9a87c-eb0e-4efd-b0d8-a4101b3a4364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:29.223691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:08:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:08:29.515408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1768c01-0fd1-4e3f-b9f3-339e18a9522d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:08:29.517095 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b1768c01-0fd1-4e3f-b9f3-339e18a9522d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:29.593993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1768c01-0fd1-4e3f-b9f3-339e18a9522d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:08:29.597784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71943c8-45fc-4a23-9186-bda5bb5d150e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:08:29.598384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71943c8-45fc-4a23-9186-bda5bb5d150e service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:29.598663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71943c8-45fc-4a23-9186-bda5bb5d150e service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:29.598936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71943c8-45fc-4a23-9186-bda5bb5d150e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:29.607851 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71943c8-45fc-4a23-9186-bda5bb5d150e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:08:29.609478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71943c8-45fc-4a23-9186-bda5bb5d150e service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:08:29.619577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 767/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:08:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:08:31.788417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77fa9d-d801-4f88-9554-847fa645e3c3 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:31.788980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77fa9d-d801-4f88-9554-847fa645e3c3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:31.788980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d77fa9d-d801-4f88-9554-847fa645e3c3 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:31.790321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:08:31.795428 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60703aca-2ab5-4d09-80f0-dbd5f0308376 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:31.795851 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60703aca-2ab5-4d09-80f0-dbd5f0308376 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:31.795969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60703aca-2ab5-4d09-80f0-dbd5f0308376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:31.816528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-60703aca-2ab5-4d09-80f0-dbd5f0308376 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:08:31.817747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-60703aca-2ab5-4d09-80f0-dbd5f0308376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:31.825476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:08:32.115341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77fa9d-d801-4f88-9554-847fa645e3c3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:08:32.126140 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d77fa9d-d801-4f88-9554-847fa645e3c3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:08:32.129641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d210e8-0cea-489a-aea9-df986500fdc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:08:32.130223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d210e8-0cea-489a-aea9-df986500fdc1 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:32.130494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d210e8-0cea-489a-aea9-df986500fdc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:32.130758 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d210e8-0cea-489a-aea9-df986500fdc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:32.139471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d210e8-0cea-489a-aea9-df986500fdc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:08:32.140883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d210e8-0cea-489a-aea9-df986500fdc1 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:08:32.150310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 765/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:08:35.968365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693053b1-a919-4a6f-b9d8-835329e29b03 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:35.968908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693053b1-a919-4a6f-b9d8-835329e29b03 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:35.968908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693053b1-a919-4a6f-b9d8-835329e29b03 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:35.970218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:08:35.975438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689d78f-626a-43b0-a7d1-71b11705910b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:35.975886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689d78f-626a-43b0-a7d1-71b11705910b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:35.975995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689d78f-626a-43b0-a7d1-71b11705910b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:35.983472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:08:35.996592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4689d78f-626a-43b0-a7d1-71b11705910b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:08:35.997802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4689d78f-626a-43b0-a7d1-71b11705910b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:36.005971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:08:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:08:36.312296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693053b1-a919-4a6f-b9d8-835329e29b03 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:08:36.314589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-693053b1-a919-4a6f-b9d8-835329e29b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:36.395267 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693053b1-a919-4a6f-b9d8-835329e29b03 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:08:36.398831 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575d8048-2267-4b0a-a950-a4b97b7d7eb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:08:36.399427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d8048-2267-4b0a-a950-a4b97b7d7eb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:36.399703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d8048-2267-4b0a-a950-a4b97b7d7eb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:36.399972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575d8048-2267-4b0a-a950-a4b97b7d7eb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:36.408814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d8048-2267-4b0a-a950-a4b97b7d7eb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:08:36.410255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575d8048-2267-4b0a-a950-a4b97b7d7eb7 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:08:36.419353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 770/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:08:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:08:38.584101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b9df-d408-479d-8e2a-decf5264839e None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:38.584641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b9df-d408-479d-8e2a-decf5264839e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:38.584641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220b9df-d408-479d-8e2a-decf5264839e None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:38.586035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 22 22:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:08:38.591360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a157-2ab3-4a87-ba11-43571bdaf0cb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:38.591794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a157-2ab3-4a87-ba11-43571bdaf0cb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:38.591888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15a157-2ab3-4a87-ba11-43571bdaf0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:38.611986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ef15a157-2ab3-4a87-ba11-43571bdaf0cb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:08:38.613250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ef15a157-2ab3-4a87-ba11-43571bdaf0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:38.620933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 22 22:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:08:38.936218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220b9df-d408-479d-8e2a-decf5264839e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:08:38.938162 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0220b9df-d408-479d-8e2a-decf5264839e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:08:38.951328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220b9df-d408-479d-8e2a-decf5264839e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:08:38.955459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748699c-dd0e-4024-8ce9-94b86ca6ddb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:08:38.956040 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748699c-dd0e-4024-8ce9-94b86ca6ddb9 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:08:38.956314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748699c-dd0e-4024-8ce9-94b86ca6ddb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:08:38.956584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748699c-dd0e-4024-8ce9-94b86ca6ddb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:08:38.965369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748699c-dd0e-4024-8ce9-94b86ca6ddb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:08:38.966805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1748699c-dd0e-4024-8ce9-94b86ca6ddb9 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:08:38.976322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 768/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:08:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:08:48.606928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:08:48.972269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=8 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:06.433229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf42199-bc0e-45bc-b996-03f476878453 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:06.434562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf42199-bc0e-45bc-b996-03f476878453 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:06.434562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf42199-bc0e-45bc-b996-03f476878453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:06.459657 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9cf42199-bc0e-45bc-b996-03f476878453 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:06.460881 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9cf42199-bc0e-45bc-b996-03f476878453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:06.473212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 772/1540] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:06.483116 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33bb9-739d-4cd5-91a1-d898183929c9 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:06.483726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33bb9-739d-4cd5-91a1-d898183929c9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:06.483726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b33bb9-739d-4cd5-91a1-d898183929c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:06.510901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-08b33bb9-739d-4cd5-91a1-d898183929c9 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:06.512118 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-08b33bb9-739d-4cd5-91a1-d898183929c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:06.525099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 769/1541] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:06.530099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf42199-bc0e-45bc-b996-03f476878453 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:06.553696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf42199-bc0e-45bc-b996-03f476878453 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:06.558848 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607c4f4-1bcf-4ff4-a0fb-57742779ce80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:06.559477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607c4f4-1bcf-4ff4-a0fb-57742779ce80 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:06.560323 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607c4f4-1bcf-4ff4-a0fb-57742779ce80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:06.560588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607c4f4-1bcf-4ff4-a0fb-57742779ce80 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:06.561524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607c4f4-1bcf-4ff4-a0fb-57742779ce80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:06.561830 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607c4f4-1bcf-4ff4-a0fb-57742779ce80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:06.562855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-a607c4f4-1bcf-4ff4-a0fb-57742779ce80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:10:06.562855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 22 22:10:06.562855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: Sep 22 22:10:06.563750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607c4f4-1bcf-4ff4-a0fb-57742779ce80 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:06.587243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 773/1542] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:06.596205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b33bb9-739d-4cd5-91a1-d898183929c9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:06.608404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b33bb9-739d-4cd5-91a1-d898183929c9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:06.615696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f625c7b-b7b0-456a-97cc-30fccd219f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:06.616315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f625c7b-b7b0-456a-97cc-30fccd219f35 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:06.616594 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f625c7b-b7b0-456a-97cc-30fccd219f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:06.616868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f625c7b-b7b0-456a-97cc-30fccd219f35 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:06.617553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f625c7b-b7b0-456a-97cc-30fccd219f35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:06.618589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-9f625c7b-b7b0-456a-97cc-30fccd219f35 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:10:06.618589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 22 22:10:06.618589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: Sep 22 22:10:06.619794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f625c7b-b7b0-456a-97cc-30fccd219f35 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:06.680637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 770/1543] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:06 2022] POST /identity/v3/projects => generated 365 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:06.685796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607c4f4-1bcf-4ff4-a0fb-57742779ce80 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:06.698965 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607c4f4-1bcf-4ff4-a0fb-57742779ce80 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:06.773754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a17a84-ef1e-4fa7-88af-f58b5dbadf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:06.774373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a17a84-ef1e-4fa7-88af-f58b5dbadf16 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:06.774836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a17a84-ef1e-4fa7-88af-f58b5dbadf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:06.775119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a17a84-ef1e-4fa7-88af-f58b5dbadf16 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:06.775959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a17a84-ef1e-4fa7-88af-f58b5dbadf16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:06.777032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-09a17a84-ef1e-4fa7-88af-f58b5dbadf16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:10:06.777032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 22 22:10:06.777032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: Sep 22 22:10:06.778296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a17a84-ef1e-4fa7-88af-f58b5dbadf16 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:06.812551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 774/1544] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:06 2022] POST /identity/v3/users => generated 283 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:10:06.816991 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f625c7b-b7b0-456a-97cc-30fccd219f35 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:06.829007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f625c7b-b7b0-456a-97cc-30fccd219f35 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:06.840530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba16f34-35de-419d-bc55-b462bcac627c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:06.841145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba16f34-35de-419d-bc55-b462bcac627c admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:06.841541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba16f34-35de-419d-bc55-b462bcac627c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:06.842019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba16f34-35de-419d-bc55-b462bcac627c admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:06.842740 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba16f34-35de-419d-bc55-b462bcac627c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:06.844402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba16f34-35de-419d-bc55-b462bcac627c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:06.864220 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 771/1545] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:06.865487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a17a84-ef1e-4fa7-88af-f58b5dbadf16 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:06.880122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a17a84-ef1e-4fa7-88af-f58b5dbadf16 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:06.886491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863279ac-a59c-4282-bc32-79846030756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:06.887146 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863279ac-a59c-4282-bc32-79846030756d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:06.887430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863279ac-a59c-4282-bc32-79846030756d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:06.887704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863279ac-a59c-4282-bc32-79846030756d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:06.888429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863279ac-a59c-4282-bc32-79846030756d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:06.890556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863279ac-a59c-4282-bc32-79846030756d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:06.896455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d8327-b0c7-49b9-9845-9bcbca082177 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:06.896813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d8327-b0c7-49b9-9845-9bcbca082177 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:06.896978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365d8327-b0c7-49b9-9845-9bcbca082177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:06.931937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-365d8327-b0c7-49b9-9845-9bcbca082177 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:06.933480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-365d8327-b0c7-49b9-9845-9bcbca082177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:06.937611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 775/1546] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:06.941791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863279ac-a59c-4282-bc32-79846030756d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:06.949295 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 772/1547] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:06.960328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863279ac-a59c-4282-bc32-79846030756d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:06.967515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9258ab-4318-466b-8f57-55d4c8a2342c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:06.967892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9258ab-4318-466b-8f57-55d4c8a2342c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:06.968179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9258ab-4318-466b-8f57-55d4c8a2342c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:06.973819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dec5a-1449-4284-9fdd-f165b6c89f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:06.974507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dec5a-1449-4284-9fdd-f165b6c89f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:06.974787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dec5a-1449-4284-9fdd-f165b6c89f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:06.975065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986dec5a-1449-4284-9fdd-f165b6c89f9c admin admin] PATH_INFO: `/v3/projects/f971cfd06e084cd8a338ba34ff27cee2/users/a71b7cf8fcf9439faba795d4bc60f5a2/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:06.996214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5b9258ab-4318-466b-8f57-55d4c8a2342c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:06.998249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5b9258ab-4318-466b-8f57-55d4c8a2342c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.009825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dec5a-1449-4284-9fdd-f165b6c89f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f971cfd06e084cd8a338ba34ff27cee2, user_id=a71b7cf8fcf9439faba795d4bc60f5a2, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.018160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 773/1548] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:07.019711 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986dec5a-1449-4284-9fdd-f165b6c89f9c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.022396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa622013-1522-4ae7-a355-754417d32e34 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.022728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa622013-1522-4ae7-a355-754417d32e34 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.023005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa622013-1522-4ae7-a355-754417d32e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.050305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 776/1549] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:06 2022] PUT /identity/v3/projects/f971cfd06e084cd8a338ba34ff27cee2/users/a71b7cf8fcf9439faba795d4bc60f5a2/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:10:07.056389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dec5a-1449-4284-9fdd-f165b6c89f9c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.072714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-aa622013-1522-4ae7-a355-754417d32e34 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.080614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-aa622013-1522-4ae7-a355-754417d32e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.213346 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986dec5a-1449-4284-9fdd-f165b6c89f9c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.219724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203b53ef-5a6e-4496-ad9f-8671bd930686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.220623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b53ef-5a6e-4496-ad9f-8671bd930686 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.221154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b53ef-5a6e-4496-ad9f-8671bd930686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.221653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203b53ef-5a6e-4496-ad9f-8671bd930686 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.221847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 774/1550] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.222824 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b53ef-5a6e-4496-ad9f-8671bd930686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:07.223201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcd5d0-c4b2-40f8-aa86-0cb0a1c24856 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.223493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcd5d0-c4b2-40f8-aa86-0cb0a1c24856 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.223771 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcd5d0-c4b2-40f8-aa86-0cb0a1c24856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.224127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b53ef-5a6e-4496-ad9f-8671bd930686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.226009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203b53ef-5a6e-4496-ad9f-8671bd930686 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.239509 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 777/1551] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:07.241297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3b3c4-f7c1-4fd4-ae60-ecf36854f8b2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.241813 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3b3c4-f7c1-4fd4-ae60-ecf36854f8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.242314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf3b3c4-f7c1-4fd4-ae60-ecf36854f8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.256755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-35fcd5d0-c4b2-40f8-aa86-0cb0a1c24856 None None] MFA Rules not processed for user `a71b7cf8fcf9439faba795d4bc60f5a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.258008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-35fcd5d0-c4b2-40f8-aa86-0cb0a1c24856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.265752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1bf3b3c4-f7c1-4fd4-ae60-ecf36854f8b2 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.267225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1bf3b3c4-f7c1-4fd4-ae60-ecf36854f8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.276916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 778/1552] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.279241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905729d8-34f8-407c-800a-7d2677f2201e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.279759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905729d8-34f8-407c-800a-7d2677f2201e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.280291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905729d8-34f8-407c-800a-7d2677f2201e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.302905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-905729d8-34f8-407c-800a-7d2677f2201e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.305543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-905729d8-34f8-407c-800a-7d2677f2201e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.316449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 779/1553] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.317899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728ab08-e377-4754-9fd5-e6a077afba14 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.318177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728ab08-e377-4754-9fd5-e6a077afba14 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.318457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1728ab08-e377-4754-9fd5-e6a077afba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.333501 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 775/1554] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.335307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244bc75-427d-4c92-b043-ad4bfe4b36ad None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.336065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244bc75-427d-4c92-b043-ad4bfe4b36ad None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.336584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244bc75-427d-4c92-b043-ad4bfe4b36ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.340910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1728ab08-e377-4754-9fd5-e6a077afba14 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.343066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1728ab08-e377-4754-9fd5-e6a077afba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.360104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 780/1555] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.360904 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728ab08-e377-4754-9fd5-e6a077afba14 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.371042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1728ab08-e377-4754-9fd5-e6a077afba14 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.374863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2244bc75-427d-4c92-b043-ad4bfe4b36ad None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.376274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2244bc75-427d-4c92-b043-ad4bfe4b36ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.376414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b0b18-73dc-4d35-be49-9cd898f6829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.376975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b0b18-73dc-4d35-be49-9cd898f6829f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.377282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b0b18-73dc-4d35-be49-9cd898f6829f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.377553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b0b18-73dc-4d35-be49-9cd898f6829f admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.378193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b0b18-73dc-4d35-be49-9cd898f6829f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.385940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b0b18-73dc-4d35-be49-9cd898f6829f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.392393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 776/1556] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.394486 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab95025-5743-412e-84ad-5c5d0dcbc394 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.394642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:07.394764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab95025-5743-412e-84ad-5c5d0dcbc394 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.394764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab95025-5743-412e-84ad-5c5d0dcbc394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.418127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9ab95025-5743-412e-84ad-5c5d0dcbc394 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.420475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9ab95025-5743-412e-84ad-5c5d0dcbc394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.427464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 781/1557] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:07.429752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e824e-481e-4d0f-9b71-2dc0a49aea28 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.430636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e824e-481e-4d0f-9b71-2dc0a49aea28 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.431102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50e824e-481e-4d0f-9b71-2dc0a49aea28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.436737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 777/1558] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.437662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab95025-5743-412e-84ad-5c5d0dcbc394 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.455209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f50e824e-481e-4d0f-9b71-2dc0a49aea28 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.455792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:07.456427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f50e824e-481e-4d0f-9b71-2dc0a49aea28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.500697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab95025-5743-412e-84ad-5c5d0dcbc394 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.524124 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 782/1559] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.525756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de232fb-9e78-459e-ba87-70dd491fd1b9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.526104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de232fb-9e78-459e-ba87-70dd491fd1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.526452 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de232fb-9e78-459e-ba87-70dd491fd1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.547468 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4de232fb-9e78-459e-ba87-70dd491fd1b9 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.548786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4de232fb-9e78-459e-ba87-70dd491fd1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.557121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23395129-6c65-43af-91fb-e32ba666f91e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.558174 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 783/1560] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.558310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23395129-6c65-43af-91fb-e32ba666f91e service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.558459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23395129-6c65-43af-91fb-e32ba666f91e service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.558790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23395129-6c65-43af-91fb-e32ba666f91e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.559386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eebdc-1c55-4c1e-916d-85a382af0dea None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.559627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eebdc-1c55-4c1e-916d-85a382af0dea None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.560604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eebdc-1c55-4c1e-916d-85a382af0dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.584689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9a3eebdc-1c55-4c1e-916d-85a382af0dea None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.585706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9a3eebdc-1c55-4c1e-916d-85a382af0dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.594325 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 784/1561] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.595534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32f728-b003-448c-afe7-527f885543df None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.595809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32f728-b003-448c-afe7-527f885543df None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.596084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32f728-b003-448c-afe7-527f885543df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.615760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9c32f728-b003-448c-afe7-527f885543df None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.617032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9c32f728-b003-448c-afe7-527f885543df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.628890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23395129-6c65-43af-91fb-e32ba666f91e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.629255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 785/1562] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.630370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23395129-6c65-43af-91fb-e32ba666f91e service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.630520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32f728-b003-448c-afe7-527f885543df None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.641349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 778/1563] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:07.642118 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23395129-6c65-43af-91fb-e32ba666f91e service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.648133 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32f728-b003-448c-afe7-527f885543df None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.654342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23395129-6c65-43af-91fb-e32ba666f91e service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.654534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd456a5-5ee5-4c6f-8b80-3a2c767d33b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.655147 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd456a5-5ee5-4c6f-8b80-3a2c767d33b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.655428 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd456a5-5ee5-4c6f-8b80-3a2c767d33b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.655698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd456a5-5ee5-4c6f-8b80-3a2c767d33b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.656350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd456a5-5ee5-4c6f-8b80-3a2c767d33b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:07.656653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd456a5-5ee5-4c6f-8b80-3a2c767d33b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.658411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd456a5-5ee5-4c6f-8b80-3a2c767d33b4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.660144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169001c6-ce6d-4a36-8b67-ab046b8d91e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.660636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169001c6-ce6d-4a36-8b67-ab046b8d91e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.661136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169001c6-ce6d-4a36-8b67-ab046b8d91e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.661513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169001c6-ce6d-4a36-8b67-ab046b8d91e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.662243 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169001c6-ce6d-4a36-8b67-ab046b8d91e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.664291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169001c6-ce6d-4a36-8b67-ab046b8d91e4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.679441 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 786/1564] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:07.679441 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd456a5-5ee5-4c6f-8b80-3a2c767d33b4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.694972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd456a5-5ee5-4c6f-8b80-3a2c767d33b4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.700673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 779/1565] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/users => generated 289 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:07.700673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169001c6-ce6d-4a36-8b67-ab046b8d91e4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.701161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b81d5-d9d5-4696-9233-e04583490e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.701278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b81d5-d9d5-4696-9233-e04583490e62 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.701622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b81d5-d9d5-4696-9233-e04583490e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.701963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b81d5-d9d5-4696-9233-e04583490e62 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.702689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b81d5-d9d5-4696-9233-e04583490e62 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:07.703064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b81d5-d9d5-4696-9233-e04583490e62 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.706683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53b81d5-d9d5-4696-9233-e04583490e62 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.708175 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169001c6-ce6d-4a36-8b67-ab046b8d91e4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.713122 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d624c-fd7d-42ed-95db-ccfb425d4b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.713728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d624c-fd7d-42ed-95db-ccfb425d4b50 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.714003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d624c-fd7d-42ed-95db-ccfb425d4b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.714269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d624c-fd7d-42ed-95db-ccfb425d4b50 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.714903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d624c-fd7d-42ed-95db-ccfb425d4b50 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:07.715201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d624c-fd7d-42ed-95db-ccfb425d4b50 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.717065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d624c-fd7d-42ed-95db-ccfb425d4b50 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.720089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 787/1566] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:07.721529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58fab-a87f-4628-9be1-dc954d8f1c41 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.721952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58fab-a87f-4628-9be1-dc954d8f1c41 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.722649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f58fab-a87f-4628-9be1-dc954d8f1c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.741149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 780/1567] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:07.742422 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d624c-fd7d-42ed-95db-ccfb425d4b50 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.748847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a9f58fab-a87f-4628-9be1-dc954d8f1c41 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.750487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a9f58fab-a87f-4628-9be1-dc954d8f1c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.762200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d624c-fd7d-42ed-95db-ccfb425d4b50 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.765250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 788/1568] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.766504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787b9ad-92ba-4c28-ac78-8e045093d276 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.766791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787b9ad-92ba-4c28-ac78-8e045093d276 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.767067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787b9ad-92ba-4c28-ac78-8e045093d276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.771502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c080bfa-f60b-4db8-bfd5-1a2cd702e4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.773017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c080bfa-f60b-4db8-bfd5-1a2cd702e4db admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.773513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c080bfa-f60b-4db8-bfd5-1a2cd702e4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.773959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c080bfa-f60b-4db8-bfd5-1a2cd702e4db admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.774612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c080bfa-f60b-4db8-bfd5-1a2cd702e4db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:07.774918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c080bfa-f60b-4db8-bfd5-1a2cd702e4db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.777299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c080bfa-f60b-4db8-bfd5-1a2cd702e4db admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.792089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 781/1569] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:07.792926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c080bfa-f60b-4db8-bfd5-1a2cd702e4db admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.799242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7787b9ad-92ba-4c28-ac78-8e045093d276 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:07.800920 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7787b9ad-92ba-4c28-ac78-8e045093d276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:07.809093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c080bfa-f60b-4db8-bfd5-1a2cd702e4db admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.809296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 789/1570] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:07.810842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787b9ad-92ba-4c28-ac78-8e045093d276 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.814581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47a979-b5c6-4af4-a6df-1f651b6f7f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.815260 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a979-b5c6-4af4-a6df-1f651b6f7f7e admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.815618 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a979-b5c6-4af4-a6df-1f651b6f7f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.815967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a979-b5c6-4af4-a6df-1f651b6f7f7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.816670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47a979-b5c6-4af4-a6df-1f651b6f7f7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.818700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47a979-b5c6-4af4-a6df-1f651b6f7f7e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.820967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787b9ad-92ba-4c28-ac78-8e045093d276 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.826248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eae88c-d430-40a4-bd0a-62904dfe55f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.826933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eae88c-d430-40a4-bd0a-62904dfe55f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.827271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eae88c-d430-40a4-bd0a-62904dfe55f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.827601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eae88c-d430-40a4-bd0a-62904dfe55f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.828339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eae88c-d430-40a4-bd0a-62904dfe55f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.830094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eae88c-d430-40a4-bd0a-62904dfe55f3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.840657 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 790/1571] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:07.844746 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eae88c-d430-40a4-bd0a-62904dfe55f3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.854485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eae88c-d430-40a4-bd0a-62904dfe55f3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.860213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 782/1572] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:07.860213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47a979-b5c6-4af4-a6df-1f651b6f7f7e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.888400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:07.917556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8db9e-0359-42ff-9f13-7cfc7b4976a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.918413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8db9e-0359-42ff-9f13-7cfc7b4976a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.918730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8db9e-0359-42ff-9f13-7cfc7b4976a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.919143 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8db9e-0359-42ff-9f13-7cfc7b4976a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.919962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8db9e-0359-42ff-9f13-7cfc7b4976a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.921993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f8db9e-0359-42ff-9f13-7cfc7b4976a4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.938792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47a979-b5c6-4af4-a6df-1f651b6f7f7e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:07.945565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168b53e-6928-4ef3-a082-43e932c7ffb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:07.946474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168b53e-6928-4ef3-a082-43e932c7ffb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:07.946925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168b53e-6928-4ef3-a082-43e932c7ffb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:07.947206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0168b53e-6928-4ef3-a082-43e932c7ffb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:07.947960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168b53e-6928-4ef3-a082-43e932c7ffb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:07.950352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0168b53e-6928-4ef3-a082-43e932c7ffb5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:07.956880 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 791/1573] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:07.958253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8db9e-0359-42ff-9f13-7cfc7b4976a4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:07.963467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:07.982540 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 783/1574] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/projects => generated 393 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:07.983635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168b53e-6928-4ef3-a082-43e932c7ffb5 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.032186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8db9e-0359-42ff-9f13-7cfc7b4976a4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.061306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0168b53e-6928-4ef3-a082-43e932c7ffb5 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.067501 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea747f-617e-4fef-b794-72293d079a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.068428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea747f-617e-4fef-b794-72293d079a6c admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.068833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea747f-617e-4fef-b794-72293d079a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.069217 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea747f-617e-4fef-b794-72293d079a6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.070251 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea747f-617e-4fef-b794-72293d079a6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.072640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea747f-617e-4fef-b794-72293d079a6c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.099665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803ba6a-e353-4c3f-80e3-3948242fc646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.100318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ba6a-e353-4c3f-80e3-3948242fc646 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.100568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ba6a-e353-4c3f-80e3-3948242fc646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.100834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803ba6a-e353-4c3f-80e3-3948242fc646 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.101637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ba6a-e353-4c3f-80e3-3948242fc646 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:08.101981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ba6a-e353-4c3f-80e3-3948242fc646 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.103552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6803ba6a-e353-4c3f-80e3-3948242fc646 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.105538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 784/1575] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:07 2022] POST /identity/v3/projects => generated 385 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:08.108104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea747f-617e-4fef-b794-72293d079a6c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.126343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 792/1576] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:08.126343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803ba6a-e353-4c3f-80e3-3948242fc646 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.136930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803ba6a-e353-4c3f-80e3-3948242fc646 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.214918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fcf5d-95a3-48f7-a6bf-334e19d96d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.216022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fcf5d-95a3-48f7-a6bf-334e19d96d2e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.216831 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fcf5d-95a3-48f7-a6bf-334e19d96d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.217302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1fcf5d-95a3-48f7-a6bf-334e19d96d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.218158 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fcf5d-95a3-48f7-a6bf-334e19d96d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.221583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1fcf5d-95a3-48f7-a6bf-334e19d96d2e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.226931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea747f-617e-4fef-b794-72293d079a6c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.239567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 793/1577] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:08.239567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fcf5d-95a3-48f7-a6bf-334e19d96d2e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.239861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cd549-d18b-4a1f-a655-d1e64de7f5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.239861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd549-d18b-4a1f-a655-d1e64de7f5ef admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.239861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd549-d18b-4a1f-a655-d1e64de7f5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.239861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18cd549-d18b-4a1f-a655-d1e64de7f5ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.240586 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cd549-d18b-4a1f-a655-d1e64de7f5ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:08.240964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cd549-d18b-4a1f-a655-d1e64de7f5ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.242956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18cd549-d18b-4a1f-a655-d1e64de7f5ef admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.253759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1fcf5d-95a3-48f7-a6bf-334e19d96d2e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.258589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 785/1578] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:08.259505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cd549-d18b-4a1f-a655-d1e64de7f5ef admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.260069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5aa823-4672-4b6c-9f65-4f9bf1db2549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.260847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aa823-4672-4b6c-9f65-4f9bf1db2549 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.261229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aa823-4672-4b6c-9f65-4f9bf1db2549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.261639 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5aa823-4672-4b6c-9f65-4f9bf1db2549 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.264764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5aa823-4672-4b6c-9f65-4f9bf1db2549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.265987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5aa823-4672-4b6c-9f65-4f9bf1db2549 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.272125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18cd549-d18b-4a1f-a655-d1e64de7f5ef admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.279438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641782f-1704-4820-9ff4-ab6ff0e65e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.280181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641782f-1704-4820-9ff4-ab6ff0e65e49 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.281388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641782f-1704-4820-9ff4-ab6ff0e65e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.281795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641782f-1704-4820-9ff4-ab6ff0e65e49 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.282856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641782f-1704-4820-9ff4-ab6ff0e65e49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.285837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641782f-1704-4820-9ff4-ab6ff0e65e49 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.305235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 794/1579] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:08 2022] POST /identity/v3/users => generated 286 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:08.305235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5aa823-4672-4b6c-9f65-4f9bf1db2549 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.313235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 786/1580] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:08 2022] POST /identity/v3/users => generated 283 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:08.313915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641782f-1704-4820-9ff4-ab6ff0e65e49 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.318494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5aa823-4672-4b6c-9f65-4f9bf1db2549 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.323675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641782f-1704-4820-9ff4-ab6ff0e65e49 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.323829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e5474-dd1b-474e-b2e8-f5e7ce85226a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.324391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e5474-dd1b-474e-b2e8-f5e7ce85226a admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.324682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e5474-dd1b-474e-b2e8-f5e7ce85226a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.324958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e5474-dd1b-474e-b2e8-f5e7ce85226a admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.325764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e5474-dd1b-474e-b2e8-f5e7ce85226a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.328604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e5474-dd1b-474e-b2e8-f5e7ce85226a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.329024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d152ce-4925-43bf-8efb-36e6c0a35fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.329848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d152ce-4925-43bf-8efb-36e6c0a35fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.330150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d152ce-4925-43bf-8efb-36e6c0a35fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.330473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d152ce-4925-43bf-8efb-36e6c0a35fd1 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.331204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d152ce-4925-43bf-8efb-36e6c0a35fd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.333127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d152ce-4925-43bf-8efb-36e6c0a35fd1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.341616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:08.364511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 795/1581] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:08 2022] POST /identity/v3/users => generated 297 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:08.364511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e5474-dd1b-474e-b2e8-f5e7ce85226a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.364823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 787/1582] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:08 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:08.364823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d152ce-4925-43bf-8efb-36e6c0a35fd1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.371954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:08.374286 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d152ce-4925-43bf-8efb-36e6c0a35fd1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.377238 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e5474-dd1b-474e-b2e8-f5e7ce85226a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.380176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcddc5-56a0-4dfa-bd2f-0ca0bd1ad7a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.382164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcddc5-56a0-4dfa-bd2f-0ca0bd1ad7a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.382449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcddc5-56a0-4dfa-bd2f-0ca0bd1ad7a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.382744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bcddc5-56a0-4dfa-bd2f-0ca0bd1ad7a7 admin admin] PATH_INFO: `/v3/projects/2a52e44d03f74727863b596c606ee35d/users/1c9c1aa9349f43e9b56ea4248c3c7f36/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.384875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d5e4f-fb6f-40e2-a4c0-4b703d6e70a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.385911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d5e4f-fb6f-40e2-a4c0-4b703d6e70a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.386395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d5e4f-fb6f-40e2-a4c0-4b703d6e70a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.386680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d5e4f-fb6f-40e2-a4c0-4b703d6e70a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.387389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d5e4f-fb6f-40e2-a4c0-4b703d6e70a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.390021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d5e4f-fb6f-40e2-a4c0-4b703d6e70a5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.399591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcddc5-56a0-4dfa-bd2f-0ca0bd1ad7a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a52e44d03f74727863b596c606ee35d, user_id=1c9c1aa9349f43e9b56ea4248c3c7f36, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.401057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-81bcddc5-56a0-4dfa-bd2f-0ca0bd1ad7a7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:10:08.401057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 22 22:10:08.401057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: Sep 22 22:10:08.403727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bcddc5-56a0-4dfa-bd2f-0ca0bd1ad7a7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.416891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 788/1583] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:08 2022] PUT /identity/v3/projects/2a52e44d03f74727863b596c606ee35d/users/1c9c1aa9349f43e9b56ea4248c3c7f36/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:08.417736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcddc5-56a0-4dfa-bd2f-0ca0bd1ad7a7 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.423635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 796/1584] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:08 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:08.423635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d5e4f-fb6f-40e2-a4c0-4b703d6e70a5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.481042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bcddc5-56a0-4dfa-bd2f-0ca0bd1ad7a7 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.486196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d5e4f-fb6f-40e2-a4c0-4b703d6e70a5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.486691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425401f-88f0-45a9-b72d-1dc96f89de0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.487000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425401f-88f0-45a9-b72d-1dc96f89de0c admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.487282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425401f-88f0-45a9-b72d-1dc96f89de0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.487554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425401f-88f0-45a9-b72d-1dc96f89de0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.488211 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425401f-88f0-45a9-b72d-1dc96f89de0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.490261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425401f-88f0-45a9-b72d-1dc96f89de0c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.491070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a566b45-e66b-4a6b-9fe5-90a99a9533ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.491680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a566b45-e66b-4a6b-9fe5-90a99a9533ee admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.491959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a566b45-e66b-4a6b-9fe5-90a99a9533ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.492235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a566b45-e66b-4a6b-9fe5-90a99a9533ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.492866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a566b45-e66b-4a6b-9fe5-90a99a9533ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.494485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a566b45-e66b-4a6b-9fe5-90a99a9533ee admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.505519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 797/1585] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:08.506325 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a566b45-e66b-4a6b-9fe5-90a99a9533ee admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.516658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a566b45-e66b-4a6b-9fe5-90a99a9533ee admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.521328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d56f80-299d-4110-8c1e-a8dfd8e8791c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.521921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56f80-299d-4110-8c1e-a8dfd8e8791c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.522203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56f80-299d-4110-8c1e-a8dfd8e8791c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.522477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d56f80-299d-4110-8c1e-a8dfd8e8791c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.526338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 789/1586] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:08 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:08.526835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d56f80-299d-4110-8c1e-a8dfd8e8791c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.526956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425401f-88f0-45a9-b72d-1dc96f89de0c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.527060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d56f80-299d-4110-8c1e-a8dfd8e8791c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.537488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 798/1587] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:08.538559 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d56f80-299d-4110-8c1e-a8dfd8e8791c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.548483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d56f80-299d-4110-8c1e-a8dfd8e8791c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.593698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425401f-88f0-45a9-b72d-1dc96f89de0c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.598507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b628b-3e77-45de-917f-a0b59aad3864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.599114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b628b-3e77-45de-917f-a0b59aad3864 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.599398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b628b-3e77-45de-917f-a0b59aad3864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.599672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b628b-3e77-45de-917f-a0b59aad3864 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.600313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b628b-3e77-45de-917f-a0b59aad3864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.601956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b628b-3e77-45de-917f-a0b59aad3864 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.604796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1b826-c0f8-47f8-adb0-97d3cdb0ef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.605437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1b826-c0f8-47f8-adb0-97d3cdb0ef40 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.605718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1b826-c0f8-47f8-adb0-97d3cdb0ef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.605993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1b826-c0f8-47f8-adb0-97d3cdb0ef40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.607188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1b826-c0f8-47f8-adb0-97d3cdb0ef40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.608745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b1b826-c0f8-47f8-adb0-97d3cdb0ef40 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.612706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 790/1588] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:08.613809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69394f0-8f9a-4017-b5e8-2ea4c6ec3781 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.614085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69394f0-8f9a-4017-b5e8-2ea4c6ec3781 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.614364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69394f0-8f9a-4017-b5e8-2ea4c6ec3781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.622027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 799/1589] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:08.622027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1b826-c0f8-47f8-adb0-97d3cdb0ef40 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.630464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1b826-c0f8-47f8-adb0-97d3cdb0ef40 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.635287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c1ddd-cb26-4cac-996c-5d78a480b2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.635878 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c1ddd-cb26-4cac-996c-5d78a480b2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.636160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c1ddd-cb26-4cac-996c-5d78a480b2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.636347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c69394f0-8f9a-4017-b5e8-2ea4c6ec3781 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:08.636460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629c1ddd-cb26-4cac-996c-5d78a480b2e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.637043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c1ddd-cb26-4cac-996c-5d78a480b2e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.637633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c69394f0-8f9a-4017-b5e8-2ea4c6ec3781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:08.639157 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629c1ddd-cb26-4cac-996c-5d78a480b2e9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.645714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 791/1590] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:08.646478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69394f0-8f9a-4017-b5e8-2ea4c6ec3781 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.659314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69394f0-8f9a-4017-b5e8-2ea4c6ec3781 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.661962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 800/1591] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:08 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:08.662817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c1ddd-cb26-4cac-996c-5d78a480b2e9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.666846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d748cb8-11a9-41da-bc4b-0ae2e5a7891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.667448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d748cb8-11a9-41da-bc4b-0ae2e5a7891a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.667733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d748cb8-11a9-41da-bc4b-0ae2e5a7891a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.668010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d748cb8-11a9-41da-bc4b-0ae2e5a7891a admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.668648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d748cb8-11a9-41da-bc4b-0ae2e5a7891a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.671296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d748cb8-11a9-41da-bc4b-0ae2e5a7891a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.673139 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629c1ddd-cb26-4cac-996c-5d78a480b2e9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.678273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2beec3-2ac0-4e3a-ad0f-fa96c730c711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.678889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2beec3-2ac0-4e3a-ad0f-fa96c730c711 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.679168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2beec3-2ac0-4e3a-ad0f-fa96c730c711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.679442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2beec3-2ac0-4e3a-ad0f-fa96c730c711 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.680064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2beec3-2ac0-4e3a-ad0f-fa96c730c711 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.681592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 792/1592] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:08.681993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2beec3-2ac0-4e3a-ad0f-fa96c730c711 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.682455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d748cb8-11a9-41da-bc4b-0ae2e5a7891a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.694041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d748cb8-11a9-41da-bc4b-0ae2e5a7891a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.698908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653a48e-f914-4922-a03d-7b75f02b6d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.699489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653a48e-f914-4922-a03d-7b75f02b6d3c admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.699767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653a48e-f914-4922-a03d-7b75f02b6d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.700058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0653a48e-f914-4922-a03d-7b75f02b6d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.700697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653a48e-f914-4922-a03d-7b75f02b6d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.702418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0653a48e-f914-4922-a03d-7b75f02b6d3c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.704832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 801/1593] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:08 2022] POST /identity/v3/users => generated 286 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:08.705485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2beec3-2ac0-4e3a-ad0f-fa96c730c711 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.716366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 793/1594] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:08.716366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653a48e-f914-4922-a03d-7b75f02b6d3c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.719288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2beec3-2ac0-4e3a-ad0f-fa96c730c711 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.724016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816d415-45b9-4c6b-97ea-9fda4683e764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.724192 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0653a48e-f914-4922-a03d-7b75f02b6d3c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.724623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d415-45b9-4c6b-97ea-9fda4683e764 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.724904 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d415-45b9-4c6b-97ea-9fda4683e764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.725210 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816d415-45b9-4c6b-97ea-9fda4683e764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.725852 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816d415-45b9-4c6b-97ea-9fda4683e764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.727437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5816d415-45b9-4c6b-97ea-9fda4683e764 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.728838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3bf47-1af5-484c-8009-a2a92367dc66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.729454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3bf47-1af5-484c-8009-a2a92367dc66 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.729734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3bf47-1af5-484c-8009-a2a92367dc66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.730005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3bf47-1af5-484c-8009-a2a92367dc66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.730629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3bf47-1af5-484c-8009-a2a92367dc66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.732859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3bf47-1af5-484c-8009-a2a92367dc66 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.742000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 802/1595] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:08.742000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816d415-45b9-4c6b-97ea-9fda4683e764 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.746181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 794/1596] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:08.746181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3bf47-1af5-484c-8009-a2a92367dc66 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.754155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3bf47-1af5-484c-8009-a2a92367dc66 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.758917 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6288d-4ed8-4dab-8059-cb13d2bd1b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.759500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6288d-4ed8-4dab-8059-cb13d2bd1b12 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.759810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6288d-4ed8-4dab-8059-cb13d2bd1b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.760083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e6288d-4ed8-4dab-8059-cb13d2bd1b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.760709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6288d-4ed8-4dab-8059-cb13d2bd1b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.761364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:08.762406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e6288d-4ed8-4dab-8059-cb13d2bd1b12 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.764565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:08.776735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 795/1597] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:08.777882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6288d-4ed8-4dab-8059-cb13d2bd1b12 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.788651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e6288d-4ed8-4dab-8059-cb13d2bd1b12 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.793537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83aa45-4c32-4201-9993-3ca7a19d66e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.794238 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83aa45-4c32-4201-9993-3ca7a19d66e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.794750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83aa45-4c32-4201-9993-3ca7a19d66e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.795065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83aa45-4c32-4201-9993-3ca7a19d66e4 admin admin] PATH_INFO: `/v3/projects/3090011abe2f47fc803b45e9aa84c70c/users/baeb24836b6f42de9c14a061c49cc941/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.800537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816d415-45b9-4c6b-97ea-9fda4683e764 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.804684 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7b538-fde7-436a-9961-78b883ebc369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.805330 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7b538-fde7-436a-9961-78b883ebc369 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.805611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7b538-fde7-436a-9961-78b883ebc369 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.805890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b7b538-fde7-436a-9961-78b883ebc369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.810483 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83aa45-4c32-4201-9993-3ca7a19d66e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3090011abe2f47fc803b45e9aa84c70c, user_id=baeb24836b6f42de9c14a061c49cc941, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.813221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83aa45-4c32-4201-9993-3ca7a19d66e4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.815883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7b538-fde7-436a-9961-78b883ebc369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.817372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b7b538-fde7-436a-9961-78b883ebc369 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.827633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 796/1598] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:08 2022] PUT /identity/v3/projects/3090011abe2f47fc803b45e9aa84c70c/users/baeb24836b6f42de9c14a061c49cc941/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:08.828616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83aa45-4c32-4201-9993-3ca7a19d66e4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.828739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 803/1599] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:08.829445 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7b538-fde7-436a-9961-78b883ebc369 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.839507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b7b538-fde7-436a-9961-78b883ebc369 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.902127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2188c-d191-48f0-86db-aee81b40c2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.902802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2188c-d191-48f0-86db-aee81b40c2f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.903128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2188c-d191-48f0-86db-aee81b40c2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.903667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe2188c-d191-48f0-86db-aee81b40c2f3 admin admin] PATH_INFO: `/v3/projects/49bb165523c54eeb87c88e83a18f6750/users/c87d7c394e04433fa73efe62473b4127/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.904454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83aa45-4c32-4201-9993-3ca7a19d66e4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.909196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bcdf2-1c6a-40a6-b1bb-5c2516c9308f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:08.909802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bcdf2-1c6a-40a6-b1bb-5c2516c9308f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:08.910074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bcdf2-1c6a-40a6-b1bb-5c2516c9308f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:08.910343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330bcdf2-1c6a-40a6-b1bb-5c2516c9308f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:08.911017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bcdf2-1c6a-40a6-b1bb-5c2516c9308f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.912576 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330bcdf2-1c6a-40a6-b1bb-5c2516c9308f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.919944 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2188c-d191-48f0-86db-aee81b40c2f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49bb165523c54eeb87c88e83a18f6750, user_id=c87d7c394e04433fa73efe62473b4127, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:08.922935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe2188c-d191-48f0-86db-aee81b40c2f3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:08.923855 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 797/1600] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:08.924972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bcdf2-1c6a-40a6-b1bb-5c2516c9308f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:08.938613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330bcdf2-1c6a-40a6-b1bb-5c2516c9308f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:08.953334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 804/1601] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:08 2022] PUT /identity/v3/projects/49bb165523c54eeb87c88e83a18f6750/users/c87d7c394e04433fa73efe62473b4127/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:08.953334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2188c-d191-48f0-86db-aee81b40c2f3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:09.010311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9896603-8cbd-4e91-a02a-234fe3050634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:09.011398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9896603-8cbd-4e91-a02a-234fe3050634 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.011857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9896603-8cbd-4e91-a02a-234fe3050634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.012288 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9896603-8cbd-4e91-a02a-234fe3050634 admin admin] PATH_INFO: `/v3/projects/4e9bfc4da4e1421e94de0946e1e5251f/users/067449d36a684dc6aa42f70f412817b4/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.022175 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe2188c-d191-48f0-86db-aee81b40c2f3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:09.027144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2730328-4088-4c4b-bb11-60112046ee51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:09.027912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730328-4088-4c4b-bb11-60112046ee51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.028630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730328-4088-4c4b-bb11-60112046ee51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.028630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2730328-4088-4c4b-bb11-60112046ee51 admin admin] PATH_INFO: `/v3/projects/0499a794df4444b594ee3126a8e210c1/users/ca01fc46c837486b808390ff0c48fa1f/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.034165 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9896603-8cbd-4e91-a02a-234fe3050634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e9bfc4da4e1421e94de0946e1e5251f, user_id=067449d36a684dc6aa42f70f412817b4, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:09.037144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9896603-8cbd-4e91-a02a-234fe3050634 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:09.048496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2730328-4088-4c4b-bb11-60112046ee51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0499a794df4444b594ee3126a8e210c1, user_id=ca01fc46c837486b808390ff0c48fa1f, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:09.055799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2730328-4088-4c4b-bb11-60112046ee51 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:09.058084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 798/1602] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:08 2022] PUT /identity/v3/projects/4e9bfc4da4e1421e94de0946e1e5251f/users/067449d36a684dc6aa42f70f412817b4/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:09.059375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9896603-8cbd-4e91-a02a-234fe3050634 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:09.085520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 805/1603] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:08 2022] PUT /identity/v3/projects/0499a794df4444b594ee3126a8e210c1/users/ca01fc46c837486b808390ff0c48fa1f/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:09.091834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab4f62-4ccd-4057-8dec-5f81e532643b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.092636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab4f62-4ccd-4057-8dec-5f81e532643b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.092918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab4f62-4ccd-4057-8dec-5f81e532643b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.125562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f8ab4f62-4ccd-4057-8dec-5f81e532643b None None] MFA Rules not processed for user `baeb24836b6f42de9c14a061c49cc941`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:09.127337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f8ab4f62-4ccd-4057-8dec-5f81e532643b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:09.129051 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:09.145001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:09.146164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9896603-8cbd-4e91-a02a-234fe3050634 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:09.219955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d748b8-0674-4d86-8fd4-ca7828bec7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:09.220768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d748b8-0674-4d86-8fd4-ca7828bec7fc admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.221254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d748b8-0674-4d86-8fd4-ca7828bec7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.221532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d748b8-0674-4d86-8fd4-ca7828bec7fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.222202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d748b8-0674-4d86-8fd4-ca7828bec7fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:09.224039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d748b8-0674-4d86-8fd4-ca7828bec7fc admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:09.232748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 806/1604] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:09.235059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab4f62-4ccd-4057-8dec-5f81e532643b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:09.243608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 799/1605] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:09.243608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9ff0-39ef-428d-882e-451c96202898 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.243608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9ff0-39ef-428d-882e-451c96202898 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.243608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9ff0-39ef-428d-882e-451c96202898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.255550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab4f62-4ccd-4057-8dec-5f81e532643b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:09.260562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6d1c3-51cf-4b56-afe7-f7ed266632ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:09.266756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6d1c3-51cf-4b56-afe7-f7ed266632ee admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.267051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6d1c3-51cf-4b56-afe7-f7ed266632ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.268081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6d1c3-51cf-4b56-afe7-f7ed266632ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.268810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6d1c3-51cf-4b56-afe7-f7ed266632ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:09.270918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6d1c3-51cf-4b56-afe7-f7ed266632ee admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:09.284272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 807/1606] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:09.285963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a808b90-acd5-431b-a39e-c51baa2286c7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.286332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a808b90-acd5-431b-a39e-c51baa2286c7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.286609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a808b90-acd5-431b-a39e-c51baa2286c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.287665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f31b9ff0-39ef-428d-882e-451c96202898 None None] MFA Rules not processed for user `c87d7c394e04433fa73efe62473b4127`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:09.288903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f31b9ff0-39ef-428d-882e-451c96202898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:09.329541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8a808b90-acd5-431b-a39e-c51baa2286c7 None None] MFA Rules not processed for user `067449d36a684dc6aa42f70f412817b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:09.333844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8a808b90-acd5-431b-a39e-c51baa2286c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:09.395521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 800/1607] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:09.399068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f51bb-b08a-4c5a-8f55-0146f8b88872 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.399068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f51bb-b08a-4c5a-8f55-0146f8b88872 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.399068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f51bb-b08a-4c5a-8f55-0146f8b88872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.420734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-da6f51bb-b08a-4c5a-8f55-0146f8b88872 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:09.422063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-da6f51bb-b08a-4c5a-8f55-0146f8b88872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:09.431923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 801/1608] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:09.433349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f51bb-b08a-4c5a-8f55-0146f8b88872 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:09.436648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 808/1609] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:09.445991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a808b90-acd5-431b-a39e-c51baa2286c7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:09.457558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f51bb-b08a-4c5a-8f55-0146f8b88872 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:09.460798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a808b90-acd5-431b-a39e-c51baa2286c7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:09.465269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19bca2-02bb-4283-baf6-8fc581d35302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:09.466890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eea430-bf22-40d1-994f-5c0fdba2fa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:09.467289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19bca2-02bb-4283-baf6-8fc581d35302 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.467601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eea430-bf22-40d1-994f-5c0fdba2fa1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.467952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eea430-bf22-40d1-994f-5c0fdba2fa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.468077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19bca2-02bb-4283-baf6-8fc581d35302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.468238 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eea430-bf22-40d1-994f-5c0fdba2fa1d admin admin] PATH_INFO: `/v3/projects/226257b343d24054bc2178b47a9a5a4f/users/bcb617f1f41c467fba9717f7f825ac80/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.468400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19bca2-02bb-4283-baf6-8fc581d35302 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.469569 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19bca2-02bb-4283-baf6-8fc581d35302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:09.472676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad19bca2-02bb-4283-baf6-8fc581d35302 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:09.488825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eea430-bf22-40d1-994f-5c0fdba2fa1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=226257b343d24054bc2178b47a9a5a4f, user_id=bcb617f1f41c467fba9717f7f825ac80, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:09.491675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eea430-bf22-40d1-994f-5c0fdba2fa1d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:09.505679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 809/1610] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:09 2022] PUT /identity/v3/projects/226257b343d24054bc2178b47a9a5a4f/users/bcb617f1f41c467fba9717f7f825ac80/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:09.506551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eea430-bf22-40d1-994f-5c0fdba2fa1d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:09.510959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 802/1611] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:09 2022] POST /identity/v3/projects => generated 371 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:09.511999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19bca2-02bb-4283-baf6-8fc581d35302 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:09.582604 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:09.586811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eea430-bf22-40d1-994f-5c0fdba2fa1d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:09.588505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad19bca2-02bb-4283-baf6-8fc581d35302 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:09.592508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a8b641-6f3c-46e6-a4e9-b5222ccdd3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:09.593195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b641-6f3c-46e6-a4e9-b5222ccdd3be admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.593475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b641-6f3c-46e6-a4e9-b5222ccdd3be admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.593747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a8b641-6f3c-46e6-a4e9-b5222ccdd3be admin admin] PATH_INFO: `/v3/projects/597b434b0f3e4187b6931397d868ca89/users/e6bc9e299582487c80d70866ef26a651/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.594526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbf970-2a37-4f88-801e-bdd9db26ed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:09.595130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbf970-2a37-4f88-801e-bdd9db26ed9e admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.595410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbf970-2a37-4f88-801e-bdd9db26ed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.595683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bbf970-2a37-4f88-801e-bdd9db26ed9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.596321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbf970-2a37-4f88-801e-bdd9db26ed9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:09.598266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bbf970-2a37-4f88-801e-bdd9db26ed9e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:09.609948 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8b641-6f3c-46e6-a4e9-b5222ccdd3be admin admin] RBAC: Authorizing `identity:create_grant(project_id=597b434b0f3e4187b6931397d868ca89, user_id=e6bc9e299582487c80d70866ef26a651, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:09.612754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a8b641-6f3c-46e6-a4e9-b5222ccdd3be admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:09.615111 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:09.630908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 803/1612] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:09 2022] POST /identity/v3/projects => generated 365 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:09.631929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 810/1613] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:09 2022] PUT /identity/v3/projects/597b434b0f3e4187b6931397d868ca89/users/e6bc9e299582487c80d70866ef26a651/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:09.632079 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbf970-2a37-4f88-801e-bdd9db26ed9e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:09.633890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44fda1-e5aa-4ab4-bb21-7f98a54f642e None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.634349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44fda1-e5aa-4ab4-bb21-7f98a54f642e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.634832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc44fda1-e5aa-4ab4-bb21-7f98a54f642e None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.642483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 811/1614] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 22 22:10:09 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:10:09.644991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927acf-8436-4c1f-9ef8-49994186ff6a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.645531 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927acf-8436-4c1f-9ef8-49994186ff6a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.645964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09927acf-8436-4c1f-9ef8-49994186ff6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.667395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-09927acf-8436-4c1f-9ef8-49994186ff6a None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:09.668614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-09927acf-8436-4c1f-9ef8-49994186ff6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:09.700242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bbf970-2a37-4f88-801e-bdd9db26ed9e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:09.706715 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45584d8b-06d2-4a29-a6fa-fad4dad2639c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:09.708104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45584d8b-06d2-4a29-a6fa-fad4dad2639c service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.708385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45584d8b-06d2-4a29-a6fa-fad4dad2639c service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.708661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45584d8b-06d2-4a29-a6fa-fad4dad2639c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.744416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 812/1615] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:09.745471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09927acf-8436-4c1f-9ef8-49994186ff6a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:09.756637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09927acf-8436-4c1f-9ef8-49994186ff6a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:09.762235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d116d3ed-47d6-4c7c-ae6a-a37f96b37771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:09.763420 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116d3ed-47d6-4c7c-ae6a-a37f96b37771 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.763764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116d3ed-47d6-4c7c-ae6a-a37f96b37771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.764047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d116d3ed-47d6-4c7c-ae6a-a37f96b37771 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.764697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116d3ed-47d6-4c7c-ae6a-a37f96b37771 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:09.766656 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d116d3ed-47d6-4c7c-ae6a-a37f96b37771 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:09.783505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45584d8b-06d2-4a29-a6fa-fad4dad2639c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:09.785091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45584d8b-06d2-4a29-a6fa-fad4dad2639c service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:09.796749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 804/1616] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:09.798196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45584d8b-06d2-4a29-a6fa-fad4dad2639c service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:09.798307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 813/1617] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:09 2022] POST /identity/v3/users => generated 292 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:09.799958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70581e27-db72-47cf-93e9-497b948333bb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.801376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70581e27-db72-47cf-93e9-497b948333bb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.801664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70581e27-db72-47cf-93e9-497b948333bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.818754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45584d8b-06d2-4a29-a6fa-fad4dad2639c service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:09.826909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f12f73-a827-450a-acf0-f724f1fd1fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:09.827674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f12f73-a827-450a-acf0-f724f1fd1fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.828547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f12f73-a827-450a-acf0-f724f1fd1fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.828547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f12f73-a827-450a-acf0-f724f1fd1fd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.829467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f12f73-a827-450a-acf0-f724f1fd1fd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:09.833540 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f12f73-a827-450a-acf0-f724f1fd1fd4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:09.862194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-70581e27-db72-47cf-93e9-497b948333bb None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:09.863444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-70581e27-db72-47cf-93e9-497b948333bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:09.874438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 805/1618] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:09 2022] POST /identity/v3/users => generated 289 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:09.874438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ccbc1-f6f5-43af-92dc-c68a1c043751 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:09.874438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ccbc1-f6f5-43af-92dc-c68a1c043751 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:09.874438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ccbc1-f6f5-43af-92dc-c68a1c043751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:09.918322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0b6ccbc1-f6f5-43af-92dc-c68a1c043751 None None] MFA Rules not processed for user `e6bc9e299582487c80d70866ef26a651`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:09.920744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0b6ccbc1-f6f5-43af-92dc-c68a1c043751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:09.993015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 814/1619] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 22 22:10:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:09.994083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70581e27-db72-47cf-93e9-497b948333bb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:10.006416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70581e27-db72-47cf-93e9-497b948333bb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:10.011541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1408c3-8823-42b7-a3b9-90c95081c785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:10.012396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1408c3-8823-42b7-a3b9-90c95081c785 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.012908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1408c3-8823-42b7-a3b9-90c95081c785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.013482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1408c3-8823-42b7-a3b9-90c95081c785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.026961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1408c3-8823-42b7-a3b9-90c95081c785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:10.029491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1408c3-8823-42b7-a3b9-90c95081c785 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:10.055678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 815/1620] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:10.058851 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1408c3-8823-42b7-a3b9-90c95081c785 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:10.084067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 806/1621] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:10.085000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ccbc1-f6f5-43af-92dc-c68a1c043751 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:10.086060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1408c3-8823-42b7-a3b9-90c95081c785 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:10.095432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f636fc-ee2d-406b-97b8-7fadeb5b5e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:10.096052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f636fc-ee2d-406b-97b8-7fadeb5b5e64 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.096337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f636fc-ee2d-406b-97b8-7fadeb5b5e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.096612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f636fc-ee2d-406b-97b8-7fadeb5b5e64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.097356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f636fc-ee2d-406b-97b8-7fadeb5b5e64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:10.099094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f636fc-ee2d-406b-97b8-7fadeb5b5e64 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:10.102896 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:10.118036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 816/1622] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:10.118820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f636fc-ee2d-406b-97b8-7fadeb5b5e64 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:10.124306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ccbc1-f6f5-43af-92dc-c68a1c043751 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:10.134438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f636fc-ee2d-406b-97b8-7fadeb5b5e64 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:10.141560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fba4c-ad4a-4216-a2dd-33dce66c4a28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:10.142208 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fba4c-ad4a-4216-a2dd-33dce66c4a28 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.142404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdac70-c20e-427f-8c50-cbd33d2037ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:10.142526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fba4c-ad4a-4216-a2dd-33dce66c4a28 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.142778 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fba4c-ad4a-4216-a2dd-33dce66c4a28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.147613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdac70-c20e-427f-8c50-cbd33d2037ce admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.147877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdac70-c20e-427f-8c50-cbd33d2037ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.148460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cdac70-c20e-427f-8c50-cbd33d2037ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.150842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cdac70-c20e-427f-8c50-cbd33d2037ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:10.152458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cdac70-c20e-427f-8c50-cbd33d2037ce admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:10.176017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 807/1623] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:10.176017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdac70-c20e-427f-8c50-cbd33d2037ce admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:10.189670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cdac70-c20e-427f-8c50-cbd33d2037ce admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:10.206634 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e22550-e6d7-460a-9b40-216db585b81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:10.207973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:10.208494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e22550-e6d7-460a-9b40-216db585b81d admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.209243 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e22550-e6d7-460a-9b40-216db585b81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.210330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e22550-e6d7-460a-9b40-216db585b81d admin admin] PATH_INFO: `/v3/projects/b6ddda7779a144a2a51e4905f4329ef4/users/afe831c11a564b04a1ec5f3d297547f0/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.237214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e22550-e6d7-460a-9b40-216db585b81d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ddda7779a144a2a51e4905f4329ef4, user_id=afe831c11a564b04a1ec5f3d297547f0, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:10.241711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e22550-e6d7-460a-9b40-216db585b81d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:10.251773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fba4c-ad4a-4216-a2dd-33dce66c4a28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:10.253974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fba4c-ad4a-4216-a2dd-33dce66c4a28 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:10.267179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 808/1624] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:10 2022] PUT /identity/v3/projects/b6ddda7779a144a2a51e4905f4329ef4/users/afe831c11a564b04a1ec5f3d297547f0/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:10.271431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66d1ae-068d-4cc5-b4b5-7c6ebfe416ee None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.271746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66d1ae-068d-4cc5-b4b5-7c6ebfe416ee None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.272967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66d1ae-068d-4cc5-b4b5-7c6ebfe416ee None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.276497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 809/1625] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 22 22:10:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:10:10.277799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66d1ae-068d-4cc5-b4b5-7c6ebfe416ee None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:10.284436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 817/1626] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:10 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:10.299579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fba4c-ad4a-4216-a2dd-33dce66c4a28 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:10.327895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fba4c-ad4a-4216-a2dd-33dce66c4a28 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:10.455273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66d1ae-068d-4cc5-b4b5-7c6ebfe416ee None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:10.467530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657a31e-e24a-470d-96cf-b3571df46c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:10.468220 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657a31e-e24a-470d-96cf-b3571df46c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.468855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832853c5-dc8f-47e8-8262-0d860c10adbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:10.469262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832853c5-dc8f-47e8-8262-0d860c10adbc admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.469762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657a31e-e24a-470d-96cf-b3571df46c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.470044 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f657a31e-e24a-470d-96cf-b3571df46c3c admin admin] PATH_INFO: `/v3/projects/74f5099bd59845158c693ae33b4fc8ce/users/ba24046dd3f24d36b1f881881fbd7354/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.470689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832853c5-dc8f-47e8-8262-0d860c10adbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.471032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832853c5-dc8f-47e8-8262-0d860c10adbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.471693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832853c5-dc8f-47e8-8262-0d860c10adbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:10.473929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832853c5-dc8f-47e8-8262-0d860c10adbc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:10.500697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 818/1627] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:10.503124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657a31e-e24a-470d-96cf-b3571df46c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f5099bd59845158c693ae33b4fc8ce, user_id=ba24046dd3f24d36b1f881881fbd7354, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:10.505649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515fcf8-c05f-4030-9d5a-f3fcf076bf32 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.507867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515fcf8-c05f-4030-9d5a-f3fcf076bf32 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.508139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f657a31e-e24a-470d-96cf-b3571df46c3c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:10.509204 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d515fcf8-c05f-4030-9d5a-f3fcf076bf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.540440 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 810/1628] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:10 2022] PUT /identity/v3/projects/74f5099bd59845158c693ae33b4fc8ce/users/ba24046dd3f24d36b1f881881fbd7354/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:10.545305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657a31e-e24a-470d-96cf-b3571df46c3c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:10.559724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d515fcf8-c05f-4030-9d5a-f3fcf076bf32 None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:10.565553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d515fcf8-c05f-4030-9d5a-f3fcf076bf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:10.695338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f657a31e-e24a-470d-96cf-b3571df46c3c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:10.711080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a379a2-b82a-4239-a40a-1403eb773af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:10.715132 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a379a2-b82a-4239-a40a-1403eb773af9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.715424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a379a2-b82a-4239-a40a-1403eb773af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.716179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a379a2-b82a-4239-a40a-1403eb773af9 admin admin] PATH_INFO: `/v3/projects/b6ddda7779a144a2a51e4905f4329ef4/users/afe831c11a564b04a1ec5f3d297547f0/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.722439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a379a2-b82a-4239-a40a-1403eb773af9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6ddda7779a144a2a51e4905f4329ef4, user_id=afe831c11a564b04a1ec5f3d297547f0, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:10.734982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a379a2-b82a-4239-a40a-1403eb773af9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:10.752666 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 811/1629] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:10 2022] PUT /identity/v3/projects/b6ddda7779a144a2a51e4905f4329ef4/users/afe831c11a564b04a1ec5f3d297547f0/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:10.754801 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a379a2-b82a-4239-a40a-1403eb773af9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:10.766513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 819/1630] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 22 22:10:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:10.778898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6d5ed-edce-4c67-8b6f-9188f5adbf41 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.779347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6d5ed-edce-4c67-8b6f-9188f5adbf41 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.779778 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6d5ed-edce-4c67-8b6f-9188f5adbf41 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.781287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 820/1631] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 22 22:10:10 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:10:10.794899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfb06e-03f2-4ee0-ac95-18871d639b62 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.795399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfb06e-03f2-4ee0-ac95-18871d639b62 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.795924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfb06e-03f2-4ee0-ac95-18871d639b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.871301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:10.880514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-eebfb06e-03f2-4ee0-ac95-18871d639b62 None None] MFA Rules not processed for user `afe831c11a564b04a1ec5f3d297547f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:10.884001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-eebfb06e-03f2-4ee0-ac95-18871d639b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:10.917136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a379a2-b82a-4239-a40a-1403eb773af9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:10.928732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bce529-11f0-4923-ad1d-0d69af2e8416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:10.929394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce529-11f0-4923-ad1d-0d69af2e8416 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:10.930950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce529-11f0-4923-ad1d-0d69af2e8416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:10.931274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce529-11f0-4923-ad1d-0d69af2e8416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:10.931944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bce529-11f0-4923-ad1d-0d69af2e8416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:10.936518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bce529-11f0-4923-ad1d-0d69af2e8416 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:10.951027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 812/1632] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:10.951027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bce529-11f0-4923-ad1d-0d69af2e8416 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:11.001073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:11.054264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bce529-11f0-4923-ad1d-0d69af2e8416 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:11.066049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2605a7-aae4-4dc5-b9da-023712d1284a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:11.067014 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2605a7-aae4-4dc5-b9da-023712d1284a service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:11.067491 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2605a7-aae4-4dc5-b9da-023712d1284a service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:11.068151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2605a7-aae4-4dc5-b9da-023712d1284a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:11.089657 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 821/1633] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:11.101557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5acc1-d484-4a9e-b768-d9335d272960 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:11.101877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5acc1-d484-4a9e-b768-d9335d272960 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:11.102157 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5acc1-d484-4a9e-b768-d9335d272960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:11.141440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5ae5acc1-d484-4a9e-b768-d9335d272960 None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:11.142769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5ae5acc1-d484-4a9e-b768-d9335d272960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:11.149632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2605a7-aae4-4dc5-b9da-023712d1284a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:11.151775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2605a7-aae4-4dc5-b9da-023712d1284a service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:11.164476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 813/1634] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:11.165603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2605a7-aae4-4dc5-b9da-023712d1284a service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:11.177382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2605a7-aae4-4dc5-b9da-023712d1284a service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:11.183062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694f22f-e831-40d6-a46c-d5ce9e64b586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:11.183916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694f22f-e831-40d6-a46c-d5ce9e64b586 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:11.186868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 822/1635] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:10:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:11.187762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5acc1-d484-4a9e-b768-d9335d272960 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:11.192512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694f22f-e831-40d6-a46c-d5ce9e64b586 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:11.193033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694f22f-e831-40d6-a46c-d5ce9e64b586 admin admin] PATH_INFO: `/v3/projects/74f5099bd59845158c693ae33b4fc8ce/users/ba24046dd3f24d36b1f881881fbd7354/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:11.195849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694f22f-e831-40d6-a46c-d5ce9e64b586 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74f5099bd59845158c693ae33b4fc8ce, user_id=ba24046dd3f24d36b1f881881fbd7354, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:11.199145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b694f22f-e831-40d6-a46c-d5ce9e64b586 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:11.218085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 814/1636] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:11 2022] PUT /identity/v3/projects/74f5099bd59845158c693ae33b4fc8ce/users/ba24046dd3f24d36b1f881881fbd7354/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:11.223046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694f22f-e831-40d6-a46c-d5ce9e64b586 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:11.303328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694f22f-e831-40d6-a46c-d5ce9e64b586 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:11.310272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df3a5c-76d8-4247-830d-26b1851241e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:11.311435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3a5c-76d8-4247-830d-26b1851241e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:11.311897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3a5c-76d8-4247-830d-26b1851241e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:11.312334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df3a5c-76d8-4247-830d-26b1851241e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:11.350374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:11.373636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5acc1-d484-4a9e-b768-d9335d272960 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:11.387315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e0386-16ba-41f3-84de-5df4e8042f79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:11.388532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0386-16ba-41f3-84de-5df4e8042f79 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:11.388816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0386-16ba-41f3-84de-5df4e8042f79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:11.391130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e0386-16ba-41f3-84de-5df4e8042f79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:11.532217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e0386-16ba-41f3-84de-5df4e8042f79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:11.533783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e0386-16ba-41f3-84de-5df4e8042f79 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:11.543440 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df3a5c-76d8-4247-830d-26b1851241e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:11.547430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df3a5c-76d8-4247-830d-26b1851241e9 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:11.556964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 823/1637] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:11.560977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39878a-ef73-4804-88c4-a7eb410655a8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:11.561289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39878a-ef73-4804-88c4-a7eb410655a8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:11.561566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39878a-ef73-4804-88c4-a7eb410655a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:11.587837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 815/1638] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:11.673510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2c39878a-ef73-4804-88c4-a7eb410655a8 None None] MFA Rules not processed for user `ba24046dd3f24d36b1f881881fbd7354`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:11.677568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2c39878a-ef73-4804-88c4-a7eb410655a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:11.789991 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df3a5c-76d8-4247-830d-26b1851241e9 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:11.789991 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df3a5c-76d8-4247-830d-26b1851241e9 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:11.808817 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1546e1f-3333-496a-8b89-e5bef8d51d6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:11.809074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1546e1f-3333-496a-8b89-e5bef8d51d6c service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:11.809444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1546e1f-3333-496a-8b89-e5bef8d51d6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:11.809694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1546e1f-3333-496a-8b89-e5bef8d51d6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:11.957834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 824/1639] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:11 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:11.969762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04745a75-e128-4164-96c9-620c22b95f02 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:11.970268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04745a75-e128-4164-96c9-620c22b95f02 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:11.970646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04745a75-e128-4164-96c9-620c22b95f02 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:11.972392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 825/1640] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 22 22:10:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:10:11.992380 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf651c0-fdfd-4e01-8ad1-9c21f465d3f1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:11.992785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf651c0-fdfd-4e01-8ad1-9c21f465d3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:11.993201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf651c0-fdfd-4e01-8ad1-9c21f465d3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:12.001265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1546e1f-3333-496a-8b89-e5bef8d51d6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:12.004714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1546e1f-3333-496a-8b89-e5bef8d51d6c service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:12.029608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 816/1641] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:10:12.030032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6bf651c0-fdfd-4e01-8ad1-9c21f465d3f1 None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:12.034504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6bf651c0-fdfd-4e01-8ad1-9c21f465d3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:12.062396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 826/1642] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:10:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:12.273978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1546e1f-3333-496a-8b89-e5bef8d51d6c service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:12.288930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1546e1f-3333-496a-8b89-e5bef8d51d6c service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:12.300216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163c0bf-8b78-4675-bec5-f16a238e1fab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:12.300968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163c0bf-8b78-4675-bec5-f16a238e1fab service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:12.301272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163c0bf-8b78-4675-bec5-f16a238e1fab service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:12.301525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b163c0bf-8b78-4675-bec5-f16a238e1fab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:12.326756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163c0bf-8b78-4675-bec5-f16a238e1fab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:12.328280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b163c0bf-8b78-4675-bec5-f16a238e1fab service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:12.349388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 817/1643] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:12.444030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf651c0-fdfd-4e01-8ad1-9c21f465d3f1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:12.464383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:12.500283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf651c0-fdfd-4e01-8ad1-9c21f465d3f1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:12.510766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fbe4b-8ba3-4927-8bdb-cac5e4930810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:12.511414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fbe4b-8ba3-4927-8bdb-cac5e4930810 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:12.511697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fbe4b-8ba3-4927-8bdb-cac5e4930810 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:12.511989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fbe4b-8ba3-4927-8bdb-cac5e4930810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:12.535180 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9fbe4b-8ba3-4927-8bdb-cac5e4930810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:12.536353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9fbe4b-8ba3-4927-8bdb-cac5e4930810 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:12.551735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 827/1644] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:13.127520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163c0bf-8b78-4675-bec5-f16a238e1fab service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:13.139560 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b163c0bf-8b78-4675-bec5-f16a238e1fab service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:13.155555 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea57f7d4-3a78-44eb-8050-eddabce71b56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:13.156267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57f7d4-3a78-44eb-8050-eddabce71b56 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:13.156617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57f7d4-3a78-44eb-8050-eddabce71b56 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:13.156965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea57f7d4-3a78-44eb-8050-eddabce71b56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:13.303584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57f7d4-3a78-44eb-8050-eddabce71b56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:13.305066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea57f7d4-3a78-44eb-8050-eddabce71b56 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:13.343045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 818/1645] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:19.704984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3741a-4eb0-438f-be14-1aaa3170ce1e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:19.705650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3741a-4eb0-438f-be14-1aaa3170ce1e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:19.705650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3741a-4eb0-438f-be14-1aaa3170ce1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:19.774608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d5e3741a-4eb0-438f-be14-1aaa3170ce1e None None] MFA Rules not processed for user `1c9c1aa9349f43e9b56ea4248c3c7f36`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:19.775891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d5e3741a-4eb0-438f-be14-1aaa3170ce1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:19.920996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355ba3c-b414-44b4-af34-73a2d21d1407 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:19.922074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355ba3c-b414-44b4-af34-73a2d21d1407 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:19.922552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d355ba3c-b414-44b4-af34-73a2d21d1407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:19.940364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 828/1646] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:19.951209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a711ea-58cd-420e-b491-4ffd255c23e6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:19.951589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a711ea-58cd-420e-b491-4ffd255c23e6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:19.952044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a711ea-58cd-420e-b491-4ffd255c23e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:20.019423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-33a711ea-58cd-420e-b491-4ffd255c23e6 None None] MFA Rules not processed for user `1c9c1aa9349f43e9b56ea4248c3c7f36`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:20.022896 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-33a711ea-58cd-420e-b491-4ffd255c23e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:20.026100 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d355ba3c-b414-44b4-af34-73a2d21d1407 None None] MFA Rules not processed for user `bcb617f1f41c467fba9717f7f825ac80`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:20.027355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d355ba3c-b414-44b4-af34-73a2d21d1407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:20.044598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 829/1647] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:20.046111 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a711ea-58cd-420e-b491-4ffd255c23e6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:20.079969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:20.082207 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a711ea-58cd-420e-b491-4ffd255c23e6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:20.099525 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa585b8-9ffe-4702-b3e4-65c840412a95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:20.100801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa585b8-9ffe-4702-b3e4-65c840412a95 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:20.101090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa585b8-9ffe-4702-b3e4-65c840412a95 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:20.101419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa585b8-9ffe-4702-b3e4-65c840412a95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:20.112955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa585b8-9ffe-4702-b3e4-65c840412a95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:20.115245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa585b8-9ffe-4702-b3e4-65c840412a95 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:20.136150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 830/1648] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:20 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:20.207938 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 819/1649] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:20.214559 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa585b8-9ffe-4702-b3e4-65c840412a95 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:20.230249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa585b8-9ffe-4702-b3e4-65c840412a95 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:20.268015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a9e49-abdd-4764-ba82-16b2f7b2a759 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:20.268667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a9e49-abdd-4764-ba82-16b2f7b2a759 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:20.270087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a9e49-abdd-4764-ba82-16b2f7b2a759 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:20.270387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a9e49-abdd-4764-ba82-16b2f7b2a759 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:20.298178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a9e49-abdd-4764-ba82-16b2f7b2a759 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:20.299794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a9e49-abdd-4764-ba82-16b2f7b2a759 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:20.312743 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 831/1650] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:20.879852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87c91c-ccf5-404a-b981-88f96d19a2cc None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:20.880337 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87c91c-ccf5-404a-b981-88f96d19a2cc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:20.880806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87c91c-ccf5-404a-b981-88f96d19a2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:20.976306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0a87c91c-ccf5-404a-b981-88f96d19a2cc None None] MFA Rules not processed for user `ca01fc46c837486b808390ff0c48fa1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:20.990382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0a87c91c-ccf5-404a-b981-88f96d19a2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:21.169857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 820/1651] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:20 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:21.190472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddded6-7afb-4f90-b8be-062674784e63 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:21.191040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddded6-7afb-4f90-b8be-062674784e63 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:21.191040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ddded6-7afb-4f90-b8be-062674784e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:21.215152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-05ddded6-7afb-4f90-b8be-062674784e63 None None] MFA Rules not processed for user `ca01fc46c837486b808390ff0c48fa1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:21.216782 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-05ddded6-7afb-4f90-b8be-062674784e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:21.225399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 832/1652] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:21.239772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87c91c-ccf5-404a-b981-88f96d19a2cc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:21.243105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937e43a-2d42-4ce6-9a12-93a0f30c590d None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:21.243394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937e43a-2d42-4ce6-9a12-93a0f30c590d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:21.243677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937e43a-2d42-4ce6-9a12-93a0f30c590d None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:21.245527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 833/1653] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 22 22:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:10:21.252680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5d4ac-7b88-4449-a902-b312c53f42da None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:21.253206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5d4ac-7b88-4449-a902-b312c53f42da None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:21.253686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5d4ac-7b88-4449-a902-b312c53f42da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:21.260102 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87c91c-ccf5-404a-b981-88f96d19a2cc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:21.268457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6756-d576-49d4-b7bc-a3664e967cd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:21.269710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6756-d576-49d4-b7bc-a3664e967cd7 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:21.270845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6756-d576-49d4-b7bc-a3664e967cd7 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:21.271569 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11a6756-d576-49d4-b7bc-a3664e967cd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:21.288767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a6756-d576-49d4-b7bc-a3664e967cd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:21.291107 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11a6756-d576-49d4-b7bc-a3664e967cd7 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:21.308592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 821/1654] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:21 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:21.345680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-33e5d4ac-7b88-4449-a902-b312c53f42da None None] MFA Rules not processed for user `18c956fa035f40fa88fbf662541121ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:21.347059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-33e5d4ac-7b88-4449-a902-b312c53f42da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:21.457434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 834/1655] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:10:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:21.473658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6756-d576-49d4-b7bc-a3664e967cd7 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:21.475215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f11a6756-d576-49d4-b7bc-a3664e967cd7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:21.502408 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11a6756-d576-49d4-b7bc-a3664e967cd7 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:21.508551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b7421-14eb-4701-b820-3c648e36dbac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:21.509332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b7421-14eb-4701-b820-3c648e36dbac service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:21.509676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b7421-14eb-4701-b820-3c648e36dbac service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:21.510013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b7421-14eb-4701-b820-3c648e36dbac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:21.526965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b7421-14eb-4701-b820-3c648e36dbac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:21.528939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b7421-14eb-4701-b820-3c648e36dbac service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:21.556769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 822/1656] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:21.842356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7314f0-8959-4dd6-8c73-ac7637b85f9b None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:21.842639 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7314f0-8959-4dd6-8c73-ac7637b85f9b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:21.842900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7314f0-8959-4dd6-8c73-ac7637b85f9b None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:21.844736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 835/1657] 10.222.0.25 () {54 vars in 912 bytes} [Thu Sep 22 22:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:10:21.857129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8da8a-5aba-4a1e-8cfe-644997fefaf5 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:21.857429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8da8a-5aba-4a1e-8cfe-644997fefaf5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:21.858646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8da8a-5aba-4a1e-8cfe-644997fefaf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:21.899179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-eaa8da8a-5aba-4a1e-8cfe-644997fefaf5 None None] MFA Rules not processed for user `18c956fa035f40fa88fbf662541121ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:21.903158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-eaa8da8a-5aba-4a1e-8cfe-644997fefaf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:21.923142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 823/1658] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:10:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:21.963123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7314f0-8959-4dd6-8c73-ac7637b85f9b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:21.979358 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7314f0-8959-4dd6-8c73-ac7637b85f9b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:21.984802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5e382-7879-4345-a7b9-82a5a2659e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:21.985073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5e382-7879-4345-a7b9-82a5a2659e14 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:21.985389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5e382-7879-4345-a7b9-82a5a2659e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:21.985671 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5e382-7879-4345-a7b9-82a5a2659e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:21.999350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5e382-7879-4345-a7b9-82a5a2659e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:22.001067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5e382-7879-4345-a7b9-82a5a2659e14 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:22.010599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 836/1659] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:22.579170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8da8a-5aba-4a1e-8cfe-644997fefaf5 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:22.723100 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5242b9-4e25-4d6f-9623-19380cc8c548 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:22.725238 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5242b9-4e25-4d6f-9623-19380cc8c548 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:22.726511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5242b9-4e25-4d6f-9623-19380cc8c548 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:22.731355 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 837/1660] 10.222.0.25 () {54 vars in 961 bytes} [Thu Sep 22 22:10:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:10:22.739661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001498-ba4f-4cd3-9c26-8b168968c266 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:22.740085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001498-ba4f-4cd3-9c26-8b168968c266 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:22.740459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b001498-ba4f-4cd3-9c26-8b168968c266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:22.776264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8da8a-5aba-4a1e-8cfe-644997fefaf5 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:22.782294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8b001498-ba4f-4cd3-9c26-8b168968c266 None None] MFA Rules not processed for user `43d6ba7b68884509a4e2859ceb72ef0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:22.783138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08750c60-7bb5-4281-9088-e6e8634ab9dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:22.783730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08750c60-7bb5-4281-9088-e6e8634ab9dd service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:22.784062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08750c60-7bb5-4281-9088-e6e8634ab9dd service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:22.784339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08750c60-7bb5-4281-9088-e6e8634ab9dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:22.786534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-08750c60-7bb5-4281-9088-e6e8634ab9dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:22.791932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8b001498-ba4f-4cd3-9c26-8b168968c266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:22.803312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08750c60-7bb5-4281-9088-e6e8634ab9dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:22.805326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08750c60-7bb5-4281-9088-e6e8634ab9dd service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:22.813964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 838/1661] 10.222.0.25 () {58 vars in 1058 bytes} [Thu Sep 22 22:10:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:22.821871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b001498-ba4f-4cd3-9c26-8b168968c266 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:22.842840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 824/1662] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:10:22.842948 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b001498-ba4f-4cd3-9c26-8b168968c266 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:22.872015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4719c5-0595-4db4-819c-aa280e62e2bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:22.873470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4719c5-0595-4db4-819c-aa280e62e2bf service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:22.874394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4719c5-0595-4db4-819c-aa280e62e2bf service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:22.875341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4719c5-0595-4db4-819c-aa280e62e2bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:22.900812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4719c5-0595-4db4-819c-aa280e62e2bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:22.902777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4719c5-0595-4db4-819c-aa280e62e2bf service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:22.941042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 839/1663] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:10:23.094218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baa805-1163-4962-94d9-2bc8228aa7cf None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:23.094536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baa805-1163-4962-94d9-2bc8228aa7cf None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:23.094815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4baa805-1163-4962-94d9-2bc8228aa7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:23.110230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:23.146191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b4baa805-1163-4962-94d9-2bc8228aa7cf None None] MFA Rules not processed for user `ca01fc46c837486b808390ff0c48fa1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:23.152922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b4baa805-1163-4962-94d9-2bc8228aa7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:23.166146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 825/1664] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:23.708260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a9b53-f12f-4ada-af16-3a6d7689641c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:23.708546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a9b53-f12f-4ada-af16-3a6d7689641c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:23.708849 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a9b53-f12f-4ada-af16-3a6d7689641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:23.721806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:23.747548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f87a9b53-f12f-4ada-af16-3a6d7689641c None None] MFA Rules not processed for user `bcb617f1f41c467fba9717f7f825ac80`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:23.750337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f87a9b53-f12f-4ada-af16-3a6d7689641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:23.774087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 840/1665] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:28.592783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baa805-1163-4962-94d9-2bc8228aa7cf None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:28.870653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4baa805-1163-4962-94d9-2bc8228aa7cf None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:28.877426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:28.878799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:28.881316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:28.881790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] PATH_INFO: `/v3/users/e6bc9e299582487c80d70866ef26a651` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:28.883398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6bc9e299582487c80d70866ef26a651)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:28.890287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: WARNING py.warnings [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:10:28.890287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: warnings.warn(msg) Sep 22 22:10:28.890287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: Sep 22 22:10:28.891991 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:29.108521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6bc9e299582487c80d70866ef26a651'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:29.109582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6bc9e299582487c80d70866ef26a651'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:29.136086 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6bc9e299582487c80d70866ef26a651'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:29.137794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6bc9e299582487c80d70866ef26a651'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:29.186997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6bc9e299582487c80d70866ef26a651'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:29.213435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 826/1666] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:28 2022] DELETE /identity/v3/users/e6bc9e299582487c80d70866ef26a651 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:29.218171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a122a4-76c7-4d6e-a89e-957d1733fb4c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:29.218691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a122a4-76c7-4d6e-a89e-957d1733fb4c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:29.218691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a122a4-76c7-4d6e-a89e-957d1733fb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:29.250343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-24a122a4-76c7-4d6e-a89e-957d1733fb4c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:29.252887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-24a122a4-76c7-4d6e-a89e-957d1733fb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:29.346634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 841/1667] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:29.357428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:29.359170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:29.441998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d2ddc-4f95-452a-9466-265fb08f1706 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:29.446833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05148e6b-2b28-445b-af0d-e8b23f6439da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:29.447447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05148e6b-2b28-445b-af0d-e8b23f6439da service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:29.447727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05148e6b-2b28-445b-af0d-e8b23f6439da service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:29.447999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05148e6b-2b28-445b-af0d-e8b23f6439da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:29.460085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05148e6b-2b28-445b-af0d-e8b23f6439da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:29.461673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05148e6b-2b28-445b-af0d-e8b23f6439da service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:29.474090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 827/1668] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:29.721507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ad034-d2ef-4745-a736-f8873b65390f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:29.721796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ad034-d2ef-4745-a736-f8873b65390f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:29.722305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ad034-d2ef-4745-a736-f8873b65390f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:29.796752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-290ad034-d2ef-4745-a736-f8873b65390f None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:29.798203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-290ad034-d2ef-4745-a736-f8873b65390f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:29.866437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05148e6b-2b28-445b-af0d-e8b23f6439da service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:29.869741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-05148e6b-2b28-445b-af0d-e8b23f6439da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:29.885274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05148e6b-2b28-445b-af0d-e8b23f6439da service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:29.922428 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 842/1669] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:10:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:29.922582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:29.923038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:29.923126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:29.923663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] PATH_INFO: `/v3/projects/597b434b0f3e4187b6931397d868ca89` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:29.926191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] RBAC: Authorizing `identity:delete_project(project_id=597b434b0f3e4187b6931397d868ca89)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:29.928246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:29.950760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:29.988688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '597b434b0f3e4187b6931397d868ca89', 'request_id': 'req-45faa741-ce0f-4d84-a0fd-8e99434a5dec'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:29.999576 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '597b434b0f3e4187b6931397d868ca89', 'request_id': 'req-45faa741-ce0f-4d84-a0fd-8e99434a5dec'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:30.008106 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '597b434b0f3e4187b6931397d868ca89', 'request_id': 'req-45faa741-ce0f-4d84-a0fd-8e99434a5dec'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:30.014744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ad034-d2ef-4745-a736-f8873b65390f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:30.016638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-290ad034-d2ef-4745-a736-f8873b65390f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:30.033639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] The token cache is being invalidate because project 597b434b0f3e4187b6931397d868ca89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:30.034025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:30.035212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-45faa741-ce0f-4d84-a0fd-8e99434a5dec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:30.037097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 828/1670] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:29 2022] DELETE /identity/v3/projects/597b434b0f3e4187b6931397d868ca89 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:30.042007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26343198-0b2f-4a22-9435-6eb4bd87ff0f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:30.042481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26343198-0b2f-4a22-9435-6eb4bd87ff0f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:30.042982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26343198-0b2f-4a22-9435-6eb4bd87ff0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:30.076086 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-26343198-0b2f-4a22-9435-6eb4bd87ff0f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:30.077521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-26343198-0b2f-4a22-9435-6eb4bd87ff0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:30.190566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:30.194709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 829/1671] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:30.202545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8679225-ab42-465d-bd0c-a23b8d541d8b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:30.202834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8679225-ab42-465d-bd0c-a23b8d541d8b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:30.203109 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8679225-ab42-465d-bd0c-a23b8d541d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:30.209381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ad034-d2ef-4745-a736-f8873b65390f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:30.227592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d8679225-ab42-465d-bd0c-a23b8d541d8b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:30.228806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d8679225-ab42-465d-bd0c-a23b8d541d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:30.243818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 830/1672] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:30.248393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8679225-ab42-465d-bd0c-a23b8d541d8b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:30.263021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8679225-ab42-465d-bd0c-a23b8d541d8b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:30.268680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da281de-5b3d-4e18-a8e1-eb638598b54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:30.269365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da281de-5b3d-4e18-a8e1-eb638598b54f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:30.269645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da281de-5b3d-4e18-a8e1-eb638598b54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:30.269925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da281de-5b3d-4e18-a8e1-eb638598b54f admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:30.270708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da281de-5b3d-4e18-a8e1-eb638598b54f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:30.271189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da281de-5b3d-4e18-a8e1-eb638598b54f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:30.272779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da281de-5b3d-4e18-a8e1-eb638598b54f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:30.292036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ae5e8-2373-41d6-8fa4-0024b968a6e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:30.293237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ae5e8-2373-41d6-8fa4-0024b968a6e4 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:30.293518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ae5e8-2373-41d6-8fa4-0024b968a6e4 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:30.293790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ae5e8-2373-41d6-8fa4-0024b968a6e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:30.298320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-237ae5e8-2373-41d6-8fa4-0024b968a6e4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:30.303346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 831/1673] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:30.304351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da281de-5b3d-4e18-a8e1-eb638598b54f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:30.315330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da281de-5b3d-4e18-a8e1-eb638598b54f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:30.321854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2d694-b2d7-44a0-959e-fc227093ddf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:30.322452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2d694-b2d7-44a0-959e-fc227093ddf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:30.322728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2d694-b2d7-44a0-959e-fc227093ddf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:30.323000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a2d694-b2d7-44a0-959e-fc227093ddf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:30.324173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2d694-b2d7-44a0-959e-fc227093ddf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:30.326598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a2d694-b2d7-44a0-959e-fc227093ddf8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:30.359328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 832/1674] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:30 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:10:30.363154 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2d694-b2d7-44a0-959e-fc227093ddf8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:30.371546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ae5e8-2373-41d6-8fa4-0024b968a6e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:30.373419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ae5e8-2373-41d6-8fa4-0024b968a6e4 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:30.381678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 843/1675] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:10:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:10:30.428155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a2d694-b2d7-44a0-959e-fc227093ddf8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:30.445343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09273295-075e-4ed1-91ee-e69fe294bf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:30.446063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09273295-075e-4ed1-91ee-e69fe294bf63 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:30.446359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09273295-075e-4ed1-91ee-e69fe294bf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:30.446648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09273295-075e-4ed1-91ee-e69fe294bf63 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:30.447295 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09273295-075e-4ed1-91ee-e69fe294bf63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:30.450393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09273295-075e-4ed1-91ee-e69fe294bf63 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:30.461658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:30.481449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 833/1676] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:30 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:10:30.487407 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ae5e8-2373-41d6-8fa4-0024b968a6e4 service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:30.499219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ae5e8-2373-41d6-8fa4-0024b968a6e4 service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:30.504127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102db8ef-b5e8-4681-9abe-36f15c148844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:30.504736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102db8ef-b5e8-4681-9abe-36f15c148844 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:30.505011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102db8ef-b5e8-4681-9abe-36f15c148844 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:30.505399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102db8ef-b5e8-4681-9abe-36f15c148844 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:30.506034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102db8ef-b5e8-4681-9abe-36f15c148844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:30.507651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102db8ef-b5e8-4681-9abe-36f15c148844 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:30.529762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 844/1677] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:30.531151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09273295-075e-4ed1-91ee-e69fe294bf63 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:30.544459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09273295-075e-4ed1-91ee-e69fe294bf63 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:30.551159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f7671-e1c3-433d-8b25-4b270209338a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:30.551783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7671-e1c3-433d-8b25-4b270209338a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:30.552061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7671-e1c3-433d-8b25-4b270209338a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:30.552457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f7671-e1c3-433d-8b25-4b270209338a admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:30.552974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f7671-e1c3-433d-8b25-4b270209338a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:30.554640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f7671-e1c3-433d-8b25-4b270209338a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:30.567248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 834/1678] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:30.571710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102db8ef-b5e8-4681-9abe-36f15c148844 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:30.583959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102db8ef-b5e8-4681-9abe-36f15c148844 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:30.589509 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9336e8-d7c0-4346-932b-ee0ca477d517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:30.590219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9336e8-d7c0-4346-932b-ee0ca477d517 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:30.590493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9336e8-d7c0-4346-932b-ee0ca477d517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:30.590799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9336e8-d7c0-4346-932b-ee0ca477d517 admin admin] PATH_INFO: `/v3/projects/09cebeea756d4e62b048c973e899b43b/users/b9a42074b397453b97fc849260c2c10b/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:30.607141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9336e8-d7c0-4346-932b-ee0ca477d517 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09cebeea756d4e62b048c973e899b43b, user_id=b9a42074b397453b97fc849260c2c10b, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:30.609863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9336e8-d7c0-4346-932b-ee0ca477d517 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:30.623744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 845/1679] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:30 2022] PUT /identity/v3/projects/09cebeea756d4e62b048c973e899b43b/users/b9a42074b397453b97fc849260c2c10b/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:30.639526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ace6dc-5bd6-4d23-82f4-e42e1bdc260a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:30.639526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ace6dc-5bd6-4d23-82f4-e42e1bdc260a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:30.639526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ace6dc-5bd6-4d23-82f4-e42e1bdc260a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:30.669824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c8ace6dc-5bd6-4d23-82f4-e42e1bdc260a None None] MFA Rules not processed for user `b9a42074b397453b97fc849260c2c10b`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:30.671321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c8ace6dc-5bd6-4d23-82f4-e42e1bdc260a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:30.740381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 835/1680] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:30.755604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9336e8-d7c0-4346-932b-ee0ca477d517 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:30.757328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bc9336e8-d7c0-4346-932b-ee0ca477d517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:30.818049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:30.835632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9336e8-d7c0-4346-932b-ee0ca477d517 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:30.842503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b82a2-dc96-4614-af53-db44cb284fa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:30.843105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b82a2-dc96-4614-af53-db44cb284fa0 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:30.843398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b82a2-dc96-4614-af53-db44cb284fa0 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:30.843663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b82a2-dc96-4614-af53-db44cb284fa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:30.851636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b82a2-dc96-4614-af53-db44cb284fa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:30.853252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b82a2-dc96-4614-af53-db44cb284fa0 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:30.862844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 846/1681] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:31.063702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ace6dc-5bd6-4d23-82f4-e42e1bdc260a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:31.125840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ace6dc-5bd6-4d23-82f4-e42e1bdc260a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:31.130253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:31.130900 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:31.131175 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:31.132405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] PATH_INFO: `/v3/users/b9a42074b397453b97fc849260c2c10b` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:31.133560 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a42074b397453b97fc849260c2c10b)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:31.135456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:31.187642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a42074b397453b97fc849260c2c10b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:31.188385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a42074b397453b97fc849260c2c10b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:31.201777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a42074b397453b97fc849260c2c10b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:31.202592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a42074b397453b97fc849260c2c10b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:31.225447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:31.227033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a42074b397453b97fc849260c2c10b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:31.246712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 836/1682] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:31 2022] DELETE /identity/v3/users/b9a42074b397453b97fc849260c2c10b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:31.251119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22d25e-24f8-48e1-b897-99109d107219 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:31.251517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22d25e-24f8-48e1-b897-99109d107219 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:31.251870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22d25e-24f8-48e1-b897-99109d107219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:31.275122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fd22d25e-24f8-48e1-b897-99109d107219 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:31.276883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fd22d25e-24f8-48e1-b897-99109d107219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:31.368838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 847/1683] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:31.379231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:31.381449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:31.449948 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508a627-ac4e-4413-8bba-78f6db147c5a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:31.453451 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c88aa45-70ff-46ac-99cc-0de9e2701bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:31.454043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88aa45-70ff-46ac-99cc-0de9e2701bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:31.454316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88aa45-70ff-46ac-99cc-0de9e2701bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:31.454583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c88aa45-70ff-46ac-99cc-0de9e2701bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:31.463165 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88aa45-70ff-46ac-99cc-0de9e2701bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:31.465125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c88aa45-70ff-46ac-99cc-0de9e2701bd1 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:31.474315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 837/1684] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:31.744665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22d25e-24f8-48e1-b897-99109d107219 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:31.746788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fd22d25e-24f8-48e1-b897-99109d107219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:31.764433 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22d25e-24f8-48e1-b897-99109d107219 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:31.769446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:31.770087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:31.770347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:31.770612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] PATH_INFO: `/v3/projects/09cebeea756d4e62b048c973e899b43b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:31.771870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09cebeea756d4e62b048c973e899b43b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:31.772960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: WARNING py.warnings [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 22 22:10:31.772960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: warnings.warn(msg) Sep 22 22:10:31.772960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: Sep 22 22:10:31.774569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:31.911237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09cebeea756d4e62b048c973e899b43b', 'request_id': 'req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:31.950389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09cebeea756d4e62b048c973e899b43b', 'request_id': 'req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:31.964658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09cebeea756d4e62b048c973e899b43b', 'request_id': 'req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:31.997538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] The token cache is being invalidate because project 09cebeea756d4e62b048c973e899b43b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:31.997538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:31.997538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a80cb1dd-cc1f-4c8d-849a-de95a125d25e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:31.998100 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 848/1685] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:31 2022] DELETE /identity/v3/projects/09cebeea756d4e62b048c973e899b43b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:32.003641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54557694-b457-4a9a-b2a2-9e5f3e1f5458 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:32.004157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54557694-b457-4a9a-b2a2-9e5f3e1f5458 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:32.004636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54557694-b457-4a9a-b2a2-9e5f3e1f5458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:32.037962 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-54557694-b457-4a9a-b2a2-9e5f3e1f5458 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:32.040473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-54557694-b457-4a9a-b2a2-9e5f3e1f5458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:32.156880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 838/1686] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:32.163793 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679a728-eb53-4df5-9f41-a1a0cc3bf6e2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:32.164081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679a728-eb53-4df5-9f41-a1a0cc3bf6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:32.164362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9679a728-eb53-4df5-9f41-a1a0cc3bf6e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:32.188758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9679a728-eb53-4df5-9f41-a1a0cc3bf6e2 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:32.190080 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9679a728-eb53-4df5-9f41-a1a0cc3bf6e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:32.202301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 849/1687] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:32.207381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54557694-b457-4a9a-b2a2-9e5f3e1f5458 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:32.221885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54557694-b457-4a9a-b2a2-9e5f3e1f5458 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:32.226610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfcf25-9c24-43ad-8e3e-1e92d8719226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:32.227225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfcf25-9c24-43ad-8e3e-1e92d8719226 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:32.227506 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfcf25-9c24-43ad-8e3e-1e92d8719226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:32.227780 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dfcf25-9c24-43ad-8e3e-1e92d8719226 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:32.233153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfcf25-9c24-43ad-8e3e-1e92d8719226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:32.233153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfcf25-9c24-43ad-8e3e-1e92d8719226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:32.233153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dfcf25-9c24-43ad-8e3e-1e92d8719226 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:32.244888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 839/1688] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:32.249615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679a728-eb53-4df5-9f41-a1a0cc3bf6e2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:32.259156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9679a728-eb53-4df5-9f41-a1a0cc3bf6e2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:32.264677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d9b4-f76a-460b-b4ba-befdfdf97abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:32.265874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d9b4-f76a-460b-b4ba-befdfdf97abb admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:32.266314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d9b4-f76a-460b-b4ba-befdfdf97abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:32.266737 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423d9b4-f76a-460b-b4ba-befdfdf97abb admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:32.267680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423d9b4-f76a-460b-b4ba-befdfdf97abb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:32.269994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6423d9b4-f76a-460b-b4ba-befdfdf97abb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:32.329305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 850/1689] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:32 2022] POST /identity/v3/projects => generated 403 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:32.330411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfcf25-9c24-43ad-8e3e-1e92d8719226 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:32.358088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dfcf25-9c24-43ad-8e3e-1e92d8719226 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:32.435159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2381a27-a37f-4d65-88ae-b958eb7dc19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:32.435801 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2381a27-a37f-4d65-88ae-b958eb7dc19f admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:32.438487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2381a27-a37f-4d65-88ae-b958eb7dc19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:32.438487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2381a27-a37f-4d65-88ae-b958eb7dc19f admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:32.438487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2381a27-a37f-4d65-88ae-b958eb7dc19f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:32.439631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2381a27-a37f-4d65-88ae-b958eb7dc19f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:32.472339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 840/1690] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:32 2022] POST /identity/v3/users => generated 302 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:32.476153 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d9b4-f76a-460b-b4ba-befdfdf97abb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:32.492768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423d9b4-f76a-460b-b4ba-befdfdf97abb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:32.499665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c88fa9-99f5-41ae-afbf-f900219540d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:32.500534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c88fa9-99f5-41ae-afbf-f900219540d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:32.501013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c88fa9-99f5-41ae-afbf-f900219540d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:32.501319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c88fa9-99f5-41ae-afbf-f900219540d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:32.502042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c88fa9-99f5-41ae-afbf-f900219540d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:32.503773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c88fa9-99f5-41ae-afbf-f900219540d5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:32.519872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 851/1691] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:10:32.534561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2381a27-a37f-4d65-88ae-b958eb7dc19f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:32.576926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2381a27-a37f-4d65-88ae-b958eb7dc19f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:32.580407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f46019-890b-458e-aea7-4a71cca79dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:32.581035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f46019-890b-458e-aea7-4a71cca79dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:32.581364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f46019-890b-458e-aea7-4a71cca79dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:32.584662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f46019-890b-458e-aea7-4a71cca79dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:32.584662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f46019-890b-458e-aea7-4a71cca79dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:32.584662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f46019-890b-458e-aea7-4a71cca79dc7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:32.587729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:32.602296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 841/1692] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:10:32.603675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c88fa9-99f5-41ae-afbf-f900219540d5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:32.615208 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c88fa9-99f5-41ae-afbf-f900219540d5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:32.629271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5a3ba-38f8-472b-af6f-15293c2bfad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:32.629895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5a3ba-38f8-472b-af6f-15293c2bfad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:32.630224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5a3ba-38f8-472b-af6f-15293c2bfad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:32.630422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5a3ba-38f8-472b-af6f-15293c2bfad4 admin admin] PATH_INFO: `/v3/projects/a54c1053450d417d833aaf8e73fe7901/users/89aad62ac3e34cb7a91403757a8c1b52/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:32.649781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5a3ba-38f8-472b-af6f-15293c2bfad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a54c1053450d417d833aaf8e73fe7901, user_id=89aad62ac3e34cb7a91403757a8c1b52, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:32.652807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5a3ba-38f8-472b-af6f-15293c2bfad4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:32.701562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 852/1693] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:32 2022] PUT /identity/v3/projects/a54c1053450d417d833aaf8e73fe7901/users/89aad62ac3e34cb7a91403757a8c1b52/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:32.703976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2039c2-e7b6-47be-b1f8-985cf528dba6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:32.704429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2039c2-e7b6-47be-b1f8-985cf528dba6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:32.704806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2039c2-e7b6-47be-b1f8-985cf528dba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:32.738741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3f2039c2-e7b6-47be-b1f8-985cf528dba6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:32.740323 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3f2039c2-e7b6-47be-b1f8-985cf528dba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:32.843669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 842/1694] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:32.853493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5a3ba-38f8-472b-af6f-15293c2bfad4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:32.855245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ffb5a3ba-38f8-472b-af6f-15293c2bfad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:32.862091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:32.953374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5a3ba-38f8-472b-af6f-15293c2bfad4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:32.962561 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc75838-d4f0-46fd-a244-ef4941c0abda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:32.963216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75838-d4f0-46fd-a244-ef4941c0abda service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:32.963795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75838-d4f0-46fd-a244-ef4941c0abda service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:32.963795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc75838-d4f0-46fd-a244-ef4941c0abda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:32.974913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc75838-d4f0-46fd-a244-ef4941c0abda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:32.976358 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc75838-d4f0-46fd-a244-ef4941c0abda service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:33.007230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 853/1695] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:10:34.533406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf152c-bc75-4633-bc79-2f25d8650df9 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:34.533938 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf152c-bc75-4633-bc79-2f25d8650df9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:34.534125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bf152c-bc75-4633-bc79-2f25d8650df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:34.624730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b009cc8-654f-4194-8841-b5fe19349d80 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:34.625892 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b009cc8-654f-4194-8841-b5fe19349d80 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:34.626418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b009cc8-654f-4194-8841-b5fe19349d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:34.663009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-02bf152c-bc75-4633-bc79-2f25d8650df9 None None] MFA Rules not processed for user `89aad62ac3e34cb7a91403757a8c1b52`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:34.670251 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-02bf152c-bc75-4633-bc79-2f25d8650df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:34.686019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7b009cc8-654f-4194-8841-b5fe19349d80 None None] MFA Rules not processed for user `18c956fa035f40fa88fbf662541121ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:34.693824 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7b009cc8-654f-4194-8841-b5fe19349d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:34.708754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 854/1696] 10.222.0.57 () {58 vars in 1015 bytes} [Thu Sep 22 22:10:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:34.712784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b009cc8-654f-4194-8841-b5fe19349d80 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:34.715007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7b009cc8-654f-4194-8841-b5fe19349d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:34.750585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b009cc8-654f-4194-8841-b5fe19349d80 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:34.758417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60e856-0654-47a2-b84f-33ecaa7d4e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:34.760071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60e856-0654-47a2-b84f-33ecaa7d4e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:34.760507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60e856-0654-47a2-b84f-33ecaa7d4e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:34.760978 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60e856-0654-47a2-b84f-33ecaa7d4e54 admin admin] PATH_INFO: `/v3/users/067449d36a684dc6aa42f70f412817b4` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:34.763288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60e856-0654-47a2-b84f-33ecaa7d4e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=067449d36a684dc6aa42f70f412817b4)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:34.765640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60e856-0654-47a2-b84f-33ecaa7d4e54 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:34.862935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 843/1697] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:34 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:34.862935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf152c-bc75-4633-bc79-2f25d8650df9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:34.894257 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bf152c-bc75-4633-bc79-2f25d8650df9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:34.907514 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43987c4-33bd-438f-8e74-4026b6a890f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:34.913518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43987c4-33bd-438f-8e74-4026b6a890f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:34.914373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7c60e856-0654-47a2-b84f-33ecaa7d4e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '067449d36a684dc6aa42f70f412817b4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:34.914528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43987c4-33bd-438f-8e74-4026b6a890f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:34.914646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43987c4-33bd-438f-8e74-4026b6a890f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:34.918582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7c60e856-0654-47a2-b84f-33ecaa7d4e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '067449d36a684dc6aa42f70f412817b4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:34.922802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7c60e856-0654-47a2-b84f-33ecaa7d4e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '067449d36a684dc6aa42f70f412817b4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:34.937317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43987c4-33bd-438f-8e74-4026b6a890f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:34.945535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43987c4-33bd-438f-8e74-4026b6a890f6 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:34.965932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7c60e856-0654-47a2-b84f-33ecaa7d4e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '067449d36a684dc6aa42f70f412817b4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:34.985074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7c60e856-0654-47a2-b84f-33ecaa7d4e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '067449d36a684dc6aa42f70f412817b4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:35.002702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 844/1698] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:35.007472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43987c4-33bd-438f-8e74-4026b6a890f6 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:35.018624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 855/1699] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:34 2022] DELETE /identity/v3/users/067449d36a684dc6aa42f70f412817b4 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:35.020784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1b9f7-1966-4d68-a38c-1ced3a2828e6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:35.021693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1b9f7-1966-4d68-a38c-1ced3a2828e6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:35.021851 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1b9f7-1966-4d68-a38c-1ced3a2828e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:35.049930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43987c4-33bd-438f-8e74-4026b6a890f6 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:35.081384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6ef1b9f7-1966-4d68-a38c-1ced3a2828e6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:35.085377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6ef1b9f7-1966-4d68-a38c-1ced3a2828e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:35.154522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c29a71-be06-4409-997f-304c598558f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:35.155507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c29a71-be06-4409-997f-304c598558f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:35.156010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c29a71-be06-4409-997f-304c598558f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:35.156693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c29a71-be06-4409-997f-304c598558f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:35.157678 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c29a71-be06-4409-997f-304c598558f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:35.160770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c29a71-be06-4409-997f-304c598558f0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:35.201322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:35.225564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 845/1700] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:35 2022] POST /identity/v3/projects => generated 405 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:35.234000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c29a71-be06-4409-997f-304c598558f0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:35.238735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-39c29a71-be06-4409-997f-304c598558f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:35.283088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 856/1701] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:35.307189 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1b9f7-1966-4d68-a38c-1ced3a2828e6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:35.308772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6ef1b9f7-1966-4d68-a38c-1ced3a2828e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:35.350514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:35.458181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c29a71-be06-4409-997f-304c598558f0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:35.464487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b1979-0570-442c-9912-ab5d7170c31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:35.465648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b1979-0570-442c-9912-ab5d7170c31c admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:35.466030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b1979-0570-442c-9912-ab5d7170c31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:35.466489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b1979-0570-442c-9912-ab5d7170c31c admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:35.467325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b1979-0570-442c-9912-ab5d7170c31c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:35.469481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b1979-0570-442c-9912-ab5d7170c31c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:35.480916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1b9f7-1966-4d68-a38c-1ced3a2828e6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:35.505552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 846/1702] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:35 2022] POST /identity/v3/users => generated 309 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:35.509763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b1979-0570-442c-9912-ab5d7170c31c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:35.557127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b1979-0570-442c-9912-ab5d7170c31c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:35.576869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1318ca7-bb5a-489f-937d-e5cded8746cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:35.577700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1318ca7-bb5a-489f-937d-e5cded8746cd admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:35.578080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1318ca7-bb5a-489f-937d-e5cded8746cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:35.578946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1318ca7-bb5a-489f-937d-e5cded8746cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:35.580508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1318ca7-bb5a-489f-937d-e5cded8746cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:35.583738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1318ca7-bb5a-489f-937d-e5cded8746cd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:35.598962 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 847/1703] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:35.605399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1318ca7-bb5a-489f-937d-e5cded8746cd admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:35.622322 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce9541-2037-4a6a-8f11-15a7a9ebc6d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:35.623392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce9541-2037-4a6a-8f11-15a7a9ebc6d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:35.623985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce9541-2037-4a6a-8f11-15a7a9ebc6d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:35.624191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce9541-2037-4a6a-8f11-15a7a9ebc6d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:35.648747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1318ca7-bb5a-489f-937d-e5cded8746cd admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:35.654208 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce9541-2037-4a6a-8f11-15a7a9ebc6d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:35.655656 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce9541-2037-4a6a-8f11-15a7a9ebc6d1 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:35.658564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d73ae7-040e-488b-beb7-23af7e0aae28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:35.659338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73ae7-040e-488b-beb7-23af7e0aae28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:35.665641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73ae7-040e-488b-beb7-23af7e0aae28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:35.666225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73ae7-040e-488b-beb7-23af7e0aae28 admin admin] PATH_INFO: `/v3/projects/7ec968a39acf4c4ab15291c604002055/users/90153a04413b4e3bb68e50a591b62abd/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:35.677000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 857/1704] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:35.700883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d73ae7-040e-488b-beb7-23af7e0aae28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec968a39acf4c4ab15291c604002055, user_id=90153a04413b4e3bb68e50a591b62abd, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:35.704231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d73ae7-040e-488b-beb7-23af7e0aae28 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:35.733976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 848/1705] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:35 2022] PUT /identity/v3/projects/7ec968a39acf4c4ab15291c604002055/users/90153a04413b4e3bb68e50a591b62abd/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:35.738229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce9541-2037-4a6a-8f11-15a7a9ebc6d1 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:35.760034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce9541-2037-4a6a-8f11-15a7a9ebc6d1 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:35.880951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56e044-34e6-4d29-8969-0f890809200b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:35.881692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56e044-34e6-4d29-8969-0f890809200b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:35.882036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56e044-34e6-4d29-8969-0f890809200b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:35.883629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56e044-34e6-4d29-8969-0f890809200b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:35.889260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d56e044-34e6-4d29-8969-0f890809200b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:35.890965 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d56e044-34e6-4d29-8969-0f890809200b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:35.913265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 858/1706] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:35 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:35.920050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d73ae7-040e-488b-beb7-23af7e0aae28 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:35.947101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d73ae7-040e-488b-beb7-23af7e0aae28 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:35.971309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c176d-fa4b-47c5-95ea-69b7bbca760a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:35.972003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c176d-fa4b-47c5-95ea-69b7bbca760a admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:35.972282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c176d-fa4b-47c5-95ea-69b7bbca760a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:35.972674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c176d-fa4b-47c5-95ea-69b7bbca760a admin admin] PATH_INFO: `/v3/projects/7ec968a39acf4c4ab15291c604002055/users/90153a04413b4e3bb68e50a591b62abd/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:35.989887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c176d-fa4b-47c5-95ea-69b7bbca760a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ec968a39acf4c4ab15291c604002055, user_id=90153a04413b4e3bb68e50a591b62abd, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:35.996204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99c176d-fa4b-47c5-95ea-69b7bbca760a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:36.072918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 849/1707] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:35 2022] PUT /identity/v3/projects/7ec968a39acf4c4ab15291c604002055/users/90153a04413b4e3bb68e50a591b62abd/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:37.134017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56e044-34e6-4d29-8969-0f890809200b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:37.135768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1d56e044-34e6-4d29-8969-0f890809200b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:37.159561 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56e044-34e6-4d29-8969-0f890809200b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:37.324556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd03de-47eb-4bf7-bcea-0b865701c7ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:37.325310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd03de-47eb-4bf7-bcea-0b865701c7ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:37.325647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd03de-47eb-4bf7-bcea-0b865701c7ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:37.325962 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd03de-47eb-4bf7-bcea-0b865701c7ab admin admin] PATH_INFO: `/v3/users/a71b7cf8fcf9439faba795d4bc60f5a2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:37.343975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd03de-47eb-4bf7-bcea-0b865701c7ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=a71b7cf8fcf9439faba795d4bc60f5a2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:37.347891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd03de-47eb-4bf7-bcea-0b865701c7ab admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:37.372113 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:37.445589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ee8a7-8de6-4b3f-b690-8d19f998ee69 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:37.446475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ee8a7-8de6-4b3f-b690-8d19f998ee69 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:37.446903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204ee8a7-8de6-4b3f-b690-8d19f998ee69 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:37.449494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 850/1708] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:10:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 22:10:37.452984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6bd03de-47eb-4bf7-bcea-0b865701c7ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a71b7cf8fcf9439faba795d4bc60f5a2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:37.453977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6bd03de-47eb-4bf7-bcea-0b865701c7ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a71b7cf8fcf9439faba795d4bc60f5a2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:37.455051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6bd03de-47eb-4bf7-bcea-0b865701c7ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a71b7cf8fcf9439faba795d4bc60f5a2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:37.473363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6bd03de-47eb-4bf7-bcea-0b865701c7ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a71b7cf8fcf9439faba795d4bc60f5a2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:37.478671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ec1d-1c3b-4b25-9508-96b6cf13c712 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:37.478978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ec1d-1c3b-4b25-9508-96b6cf13c712 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:37.480670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c1ec1d-1c3b-4b25-9508-96b6cf13c712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:37.484125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6bd03de-47eb-4bf7-bcea-0b865701c7ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a71b7cf8fcf9439faba795d4bc60f5a2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:37.517589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 859/1709] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:37 2022] DELETE /identity/v3/users/a71b7cf8fcf9439faba795d4bc60f5a2 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:37.519335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754bad1c-cdd8-4db9-b0e8-2bece7dfb39e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:37.521991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754bad1c-cdd8-4db9-b0e8-2bece7dfb39e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:37.522553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754bad1c-cdd8-4db9-b0e8-2bece7dfb39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:37.530671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b1c1ec1d-1c3b-4b25-9508-96b6cf13c712 None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:37.534577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b1c1ec1d-1c3b-4b25-9508-96b6cf13c712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:37.583774 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-754bad1c-cdd8-4db9-b0e8-2bece7dfb39e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:37.587901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-754bad1c-cdd8-4db9-b0e8-2bece7dfb39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:37.754613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 851/1710] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:10:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:37.767159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 860/1711] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:37.794367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1ec1d-1c3b-4b25-9508-96b6cf13c712 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:37.796070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b1c1ec1d-1c3b-4b25-9508-96b6cf13c712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:37.816976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c1ec1d-1c3b-4b25-9508-96b6cf13c712 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:37.829091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:37.830019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:37.830391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:37.830754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] PATH_INFO: `/v3/users/c87d7c394e04433fa73efe62473b4127` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:37.832074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c87d7c394e04433fa73efe62473b4127)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:37.834474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:37.900153 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754bad1c-cdd8-4db9-b0e8-2bece7dfb39e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:37.904874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-754bad1c-cdd8-4db9-b0e8-2bece7dfb39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:37.906393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:37.985588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c87d7c394e04433fa73efe62473b4127'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:37.985866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c87d7c394e04433fa73efe62473b4127'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:38.014399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c87d7c394e04433fa73efe62473b4127'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:38.017500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c87d7c394e04433fa73efe62473b4127'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:38.118885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c87d7c394e04433fa73efe62473b4127'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:38.180372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 852/1712] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:37 2022] DELETE /identity/v3/users/c87d7c394e04433fa73efe62473b4127 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:38.181341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:38.188647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754bad1c-cdd8-4db9-b0e8-2bece7dfb39e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:38.190089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:38.266177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:38.312879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea542ee-4521-4a25-9cf6-d9d7fcbe9904 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:38.316298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea542ee-4521-4a25-9cf6-d9d7fcbe9904 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:38.316758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea542ee-4521-4a25-9cf6-d9d7fcbe9904 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:38.318120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea542ee-4521-4a25-9cf6-d9d7fcbe9904 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:38.322757 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2ea542ee-4521-4a25-9cf6-d9d7fcbe9904 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:38.481403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867d864-aa54-4b17-b81d-9dc8f9d47204 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:38.512669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff6971-6175-482f-9bed-c6b8834681fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:38.513485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff6971-6175-482f-9bed-c6b8834681fd service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:38.513770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff6971-6175-482f-9bed-c6b8834681fd service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:38.514041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccff6971-6175-482f-9bed-c6b8834681fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:38.518092 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ccff6971-6175-482f-9bed-c6b8834681fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:38.525772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea542ee-4521-4a25-9cf6-d9d7fcbe9904 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:38.529663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea542ee-4521-4a25-9cf6-d9d7fcbe9904 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:38.570106 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff6971-6175-482f-9bed-c6b8834681fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:38.585938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 861/1713] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:38.586125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccff6971-6175-482f-9bed-c6b8834681fd service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:38.587365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3b5b4-889b-4dde-a701-16d9381412bb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:38.587715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3b5b4-889b-4dde-a701-16d9381412bb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:38.587993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3b5b4-889b-4dde-a701-16d9381412bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:38.611888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 853/1714] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:38.663436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4fd3b5b4-889b-4dde-a701-16d9381412bb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:38.664694 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4fd3b5b4-889b-4dde-a701-16d9381412bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:38.750367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 862/1715] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:38.819214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff6971-6175-482f-9bed-c6b8834681fd service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:38.820659 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ccff6971-6175-482f-9bed-c6b8834681fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:38.846307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccff6971-6175-482f-9bed-c6b8834681fd service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:38.867584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13fcd5-d164-4d36-ad2b-354b69081d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:38.867584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13fcd5-d164-4d36-ad2b-354b69081d06 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:38.867584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13fcd5-d164-4d36-ad2b-354b69081d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:38.867584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13fcd5-d164-4d36-ad2b-354b69081d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:38.901374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13fcd5-d164-4d36-ad2b-354b69081d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:38.904165 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13fcd5-d164-4d36-ad2b-354b69081d06 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:38.919904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 854/1716] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:39.532922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3b5b4-889b-4dde-a701-16d9381412bb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:39.534656 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4fd3b5b4-889b-4dde-a701-16d9381412bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:39.576623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3b5b4-889b-4dde-a701-16d9381412bb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:39.585876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:39.587501 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:39.588403 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:39.590275 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] PATH_INFO: `/v3/projects/f971cfd06e084cd8a338ba34ff27cee2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:39.594049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f971cfd06e084cd8a338ba34ff27cee2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:39.597724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:39.680920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13fcd5-d164-4d36-ad2b-354b69081d06 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:39.683666 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5a13fcd5-d164-4d36-ad2b-354b69081d06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:39.709911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13fcd5-d164-4d36-ad2b-354b69081d06 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:39.717045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f971cfd06e084cd8a338ba34ff27cee2', 'request_id': 'req-c582668a-849c-4883-b9dd-faa66df089f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:39.726413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f971cfd06e084cd8a338ba34ff27cee2', 'request_id': 'req-c582668a-849c-4883-b9dd-faa66df089f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:39.738377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f971cfd06e084cd8a338ba34ff27cee2', 'request_id': 'req-c582668a-849c-4883-b9dd-faa66df089f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:39.787544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] The token cache is being invalidate because project f971cfd06e084cd8a338ba34ff27cee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:39.787838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:39.789333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c582668a-849c-4883-b9dd-faa66df089f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:39.791293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 863/1717] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:39 2022] DELETE /identity/v3/projects/f971cfd06e084cd8a338ba34ff27cee2 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:39.796929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4a509-4e65-40dd-b212-52a1ac96f5d4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:39.797284 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4a509-4e65-40dd-b212-52a1ac96f5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:39.797557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4a509-4e65-40dd-b212-52a1ac96f5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:39.821476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:39.823282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:39.836030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:39.836488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] PATH_INFO: `/v3/projects/4e9bfc4da4e1421e94de0946e1e5251f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:39.841922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e9bfc4da4e1421e94de0946e1e5251f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:39.851147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:39.874319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-89b4a509-4e65-40dd-b212-52a1ac96f5d4 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:39.875573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-89b4a509-4e65-40dd-b212-52a1ac96f5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:39.895741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 864/1718] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:39.905430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4b56f-7051-44a4-b423-b6877d59fb95 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:39.905820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4b56f-7051-44a4-b423-b6877d59fb95 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:39.906118 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4b56f-7051-44a4-b423-b6877d59fb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:39.928353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:39.943594 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1bd4b56f-7051-44a4-b423-b6877d59fb95 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:39.946202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1bd4b56f-7051-44a4-b423-b6877d59fb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:39.961678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 865/1719] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:39.962796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4b56f-7051-44a4-b423-b6877d59fb95 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:39.978563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e9bfc4da4e1421e94de0946e1e5251f', 'request_id': 'req-fc6a61fa-85fd-46db-bfd3-ece06d17c100'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:39.987065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4b56f-7051-44a4-b423-b6877d59fb95 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:39.997965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e9bfc4da4e1421e94de0946e1e5251f', 'request_id': 'req-fc6a61fa-85fd-46db-bfd3-ece06d17c100'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:40.054088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e9bfc4da4e1421e94de0946e1e5251f', 'request_id': 'req-fc6a61fa-85fd-46db-bfd3-ece06d17c100'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:40.106104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] The token cache is being invalidate because project 4e9bfc4da4e1421e94de0946e1e5251f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:40.106104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:40.111359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:40.112148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fc6a61fa-85fd-46db-bfd3-ece06d17c100 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:40.126341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 855/1720] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:39 2022] DELETE /identity/v3/projects/4e9bfc4da4e1421e94de0946e1e5251f => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:40.131892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5da99-9727-4b4e-bb62-8fbc8facd0e4 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.132196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5da99-9727-4b4e-bb62-8fbc8facd0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.132480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef5da99-9727-4b4e-bb62-8fbc8facd0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.144353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac264a-cb7f-4c82-95bd-9117e0f921b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:40.144856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac264a-cb7f-4c82-95bd-9117e0f921b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.145166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac264a-cb7f-4c82-95bd-9117e0f921b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.145444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac264a-cb7f-4c82-95bd-9117e0f921b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.146100 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac264a-cb7f-4c82-95bd-9117e0f921b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:40.146399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac264a-cb7f-4c82-95bd-9117e0f921b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:40.156800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac264a-cb7f-4c82-95bd-9117e0f921b2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:40.188225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1ef5da99-9727-4b4e-bb62-8fbc8facd0e4 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:40.189505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1ef5da99-9727-4b4e-bb62-8fbc8facd0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:40.232000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 866/1721] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:40.232936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac264a-cb7f-4c82-95bd-9117e0f921b2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:40.235207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 856/1722] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:40.242695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c7ac264a-cb7f-4c82-95bd-9117e0f921b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:40.244458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46b9ec-0767-469c-b2f1-a5f3b69ea892 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.246212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46b9ec-0767-469c-b2f1-a5f3b69ea892 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.249026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46b9ec-0767-469c-b2f1-a5f3b69ea892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.257944 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac264a-cb7f-4c82-95bd-9117e0f921b2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:40.265637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722801b7-16ff-44f7-9a2b-4220c5b82219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:40.267301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722801b7-16ff-44f7-9a2b-4220c5b82219 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.267607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722801b7-16ff-44f7-9a2b-4220c5b82219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.268064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722801b7-16ff-44f7-9a2b-4220c5b82219 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.268714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722801b7-16ff-44f7-9a2b-4220c5b82219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:40.270755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722801b7-16ff-44f7-9a2b-4220c5b82219 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:40.318452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-cf46b9ec-0767-469c-b2f1-a5f3b69ea892 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:40.320504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cf46b9ec-0767-469c-b2f1-a5f3b69ea892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:40.329574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 867/1723] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:40 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:40.330794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722801b7-16ff-44f7-9a2b-4220c5b82219 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:40.333069 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 857/1724] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:40.338026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46b9ec-0767-469c-b2f1-a5f3b69ea892 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:40.357752 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46b9ec-0767-469c-b2f1-a5f3b69ea892 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:40.448592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071cca8-d582-40a9-b139-2c7c4753236f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:40.450251 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071cca8-d582-40a9-b139-2c7c4753236f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.450585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071cca8-d582-40a9-b139-2c7c4753236f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.450922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071cca8-d582-40a9-b139-2c7c4753236f admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.451653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071cca8-d582-40a9-b139-2c7c4753236f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:40.451987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071cca8-d582-40a9-b139-2c7c4753236f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:40.453813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071cca8-d582-40a9-b139-2c7c4753236f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:40.469215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722801b7-16ff-44f7-9a2b-4220c5b82219 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:40.469445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 858/1725] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:40.475070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071cca8-d582-40a9-b139-2c7c4753236f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:40.477613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f34d0-8c07-480b-930a-f1fcbfb3d514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:40.478241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f34d0-8c07-480b-930a-f1fcbfb3d514 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.480735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f34d0-8c07-480b-930a-f1fcbfb3d514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.481023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196f34d0-8c07-480b-930a-f1fcbfb3d514 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.482561 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f34d0-8c07-480b-930a-f1fcbfb3d514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:40.488121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196f34d0-8c07-480b-930a-f1fcbfb3d514 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:40.500273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071cca8-d582-40a9-b139-2c7c4753236f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:40.512093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206568b3-30de-4bb4-a9b3-dfcbf88cfa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:40.513913 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206568b3-30de-4bb4-a9b3-dfcbf88cfa10 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.514466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206568b3-30de-4bb4-a9b3-dfcbf88cfa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.514466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206568b3-30de-4bb4-a9b3-dfcbf88cfa10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.516528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206568b3-30de-4bb4-a9b3-dfcbf88cfa10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:40.522149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206568b3-30de-4bb4-a9b3-dfcbf88cfa10 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:40.554087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 868/1726] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:40 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:40.554087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f34d0-8c07-480b-930a-f1fcbfb3d514 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:40.571416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196f34d0-8c07-480b-930a-f1fcbfb3d514 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:40.583127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdde34-070a-43dd-bfbf-c1fdb2c7b476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:40.584804 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdde34-070a-43dd-bfbf-c1fdb2c7b476 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.587082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdde34-070a-43dd-bfbf-c1fdb2c7b476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.587380 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdde34-070a-43dd-bfbf-c1fdb2c7b476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.588092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdde34-070a-43dd-bfbf-c1fdb2c7b476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:40.592826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdde34-070a-43dd-bfbf-c1fdb2c7b476 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:40.613925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 859/1727] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:40 2022] POST /identity/v3/projects => generated 401 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:40.618021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206568b3-30de-4bb4-a9b3-dfcbf88cfa10 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:40.637514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 869/1728] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:40.641348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdde34-070a-43dd-bfbf-c1fdb2c7b476 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:40.670221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdde34-070a-43dd-bfbf-c1fdb2c7b476 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:40.742589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206568b3-30de-4bb4-a9b3-dfcbf88cfa10 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:40.754539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da85e5b-b270-43cb-8516-cc0696933ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:40.755714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da85e5b-b270-43cb-8516-cc0696933ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.756288 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da85e5b-b270-43cb-8516-cc0696933ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.756737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da85e5b-b270-43cb-8516-cc0696933ad6 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.758011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da85e5b-b270-43cb-8516-cc0696933ad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:40.762124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da85e5b-b270-43cb-8516-cc0696933ad6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:40.782781 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:40.802229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049587f-e221-486d-90c7-5e1132c0cbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:40.803198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049587f-e221-486d-90c7-5e1132c0cbfc admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.803845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049587f-e221-486d-90c7-5e1132c0cbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.804308 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a049587f-e221-486d-90c7-5e1132c0cbfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.805695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049587f-e221-486d-90c7-5e1132c0cbfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:40.809667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a049587f-e221-486d-90c7-5e1132c0cbfc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:40.810356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 860/1729] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:40 2022] POST /identity/v3/users => generated 301 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:40.813323 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da85e5b-b270-43cb-8516-cc0696933ad6 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:40.833251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 870/1730] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:10:40.838082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049587f-e221-486d-90c7-5e1132c0cbfc admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:40.846004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da85e5b-b270-43cb-8516-cc0696933ad6 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:40.856498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a049587f-e221-486d-90c7-5e1132c0cbfc admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:40.859954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ec211-c6bb-4ae0-b464-51f8e8861523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:40.861664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ec211-c6bb-4ae0-b464-51f8e8861523 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.863772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ec211-c6bb-4ae0-b464-51f8e8861523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.863772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ec211-c6bb-4ae0-b464-51f8e8861523 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.864666 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ec211-c6bb-4ae0-b464-51f8e8861523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:40.865044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccd60c-86f3-40d6-a942-2b9501d53b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:40.866633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccd60c-86f3-40d6-a942-2b9501d53b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.867068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccd60c-86f3-40d6-a942-2b9501d53b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.867492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccd60c-86f3-40d6-a942-2b9501d53b6e admin admin] PATH_INFO: `/v3/projects/a2853b8131e448f7abe704f93a27e3dc/users/5badae31e2384b919c04264f670a17a3/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:40.871490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ec211-c6bb-4ae0-b464-51f8e8861523 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:40.899395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccd60c-86f3-40d6-a942-2b9501d53b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2853b8131e448f7abe704f93a27e3dc, user_id=5badae31e2384b919c04264f670a17a3, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:40.903108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 861/1731] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:40.907651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ec211-c6bb-4ae0-b464-51f8e8861523 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:40.908473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccd60c-86f3-40d6-a942-2b9501d53b6e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:40.953772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 871/1732] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:40 2022] PUT /identity/v3/projects/a2853b8131e448f7abe704f93a27e3dc/users/5badae31e2384b919c04264f670a17a3/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:10:40.958885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887419c-c563-44ed-a865-86392694576b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:40.959176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887419c-c563-44ed-a865-86392694576b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:40.960208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ec211-c6bb-4ae0-b464-51f8e8861523 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:40.960875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f887419c-c563-44ed-a865-86392694576b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:41.046217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f887419c-c563-44ed-a865-86392694576b None None] MFA Rules not processed for user `5badae31e2384b919c04264f670a17a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:41.047615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f887419c-c563-44ed-a865-86392694576b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:41.058199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:41.136736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc52b7-d1ec-42fd-8a10-ac51ed483637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:41.137842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc52b7-d1ec-42fd-8a10-ac51ed483637 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:41.138315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc52b7-d1ec-42fd-8a10-ac51ed483637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:41.138826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc52b7-d1ec-42fd-8a10-ac51ed483637 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:41.141453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc52b7-d1ec-42fd-8a10-ac51ed483637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:41.145866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc52b7-d1ec-42fd-8a10-ac51ed483637 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:41.173426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 862/1733] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:40 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:41.177460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc52b7-d1ec-42fd-8a10-ac51ed483637 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:41.217548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc52b7-d1ec-42fd-8a10-ac51ed483637 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:41.230375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8996de9-b864-49f1-8547-71d94699a8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:41.232327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8996de9-b864-49f1-8547-71d94699a8ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:41.232974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8996de9-b864-49f1-8547-71d94699a8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:41.234173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8996de9-b864-49f1-8547-71d94699a8ca admin admin] PATH_INFO: `/v3/projects/ba3d716a89394d21a5c772aae6d6472d/users/af7fb86fba9b49c3891894dda31e8e89/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:41.273822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8996de9-b864-49f1-8547-71d94699a8ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba3d716a89394d21a5c772aae6d6472d, user_id=af7fb86fba9b49c3891894dda31e8e89, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:41.276616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8996de9-b864-49f1-8547-71d94699a8ca admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:41.302048 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 872/1734] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:41.306268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887419c-c563-44ed-a865-86392694576b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:41.328219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f887419c-c563-44ed-a865-86392694576b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:41.333577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 863/1735] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:41 2022] PUT /identity/v3/projects/ba3d716a89394d21a5c772aae6d6472d/users/af7fb86fba9b49c3891894dda31e8e89/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:10:41.349121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11a05-ae0c-416e-a7a1-64c7c3c65671 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:41.349432 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11a05-ae0c-416e-a7a1-64c7c3c65671 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:41.349730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb11a05-ae0c-416e-a7a1-64c7c3c65671 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:41.508554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8eb11a05-ae0c-416e-a7a1-64c7c3c65671 None None] MFA Rules not processed for user `af7fb86fba9b49c3891894dda31e8e89`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:41.509887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8eb11a05-ae0c-416e-a7a1-64c7c3c65671 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:41.518360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ce74f-790e-488f-ac91-dc26b98b58d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:41.519125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ce74f-790e-488f-ac91-dc26b98b58d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:41.519364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ce74f-790e-488f-ac91-dc26b98b58d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:41.519645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ce74f-790e-488f-ac91-dc26b98b58d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:41.520301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ce74f-790e-488f-ac91-dc26b98b58d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:41.527524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ce74f-790e-488f-ac91-dc26b98b58d7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:41.580497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 873/1736] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:41 2022] POST /identity/v3/projects => generated 377 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:10:41.584951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ce74f-790e-488f-ac91-dc26b98b58d7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:41.697406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 864/1737] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:41 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:41.699606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fad234-3f3e-4314-ba71-2eb0361b60fa None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:41.699890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fad234-3f3e-4314-ba71-2eb0361b60fa None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:41.700164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fad234-3f3e-4314-ba71-2eb0361b60fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:41.744273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ce74f-790e-488f-ac91-dc26b98b58d7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:41.754053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13fb49-a978-4aac-804e-c231870a59d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:41.755485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13fb49-a978-4aac-804e-c231870a59d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:41.755772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13fb49-a978-4aac-804e-c231870a59d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:41.756100 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13fb49-a978-4aac-804e-c231870a59d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:41.756761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13fb49-a978-4aac-804e-c231870a59d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:41.762054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13fb49-a978-4aac-804e-c231870a59d5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:41.772997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f4fad234-3f3e-4314-ba71-2eb0361b60fa None None] MFA Rules not processed for user `90153a04413b4e3bb68e50a591b62abd`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:41.775348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f4fad234-3f3e-4314-ba71-2eb0361b60fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:41.822104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 874/1738] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:41 2022] POST /identity/v3/users => generated 295 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:41.824894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13fb49-a978-4aac-804e-c231870a59d5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:41.849766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13fb49-a978-4aac-804e-c231870a59d5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:41.861871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439dc4f-8f99-4688-b54e-a5a6f57053ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:41.863092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439dc4f-8f99-4688-b54e-a5a6f57053ec admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:41.863092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439dc4f-8f99-4688-b54e-a5a6f57053ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:41.864681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e439dc4f-8f99-4688-b54e-a5a6f57053ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:41.865366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439dc4f-8f99-4688-b54e-a5a6f57053ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:41.874240 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e439dc4f-8f99-4688-b54e-a5a6f57053ec admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:41.879598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:41.942905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 875/1739] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:41 2022] POST /identity/v3/projects => generated 399 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:41.943891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439dc4f-8f99-4688-b54e-a5a6f57053ec admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:41.957315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:41.973819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 865/1740] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:41 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:41.973819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fad234-3f3e-4314-ba71-2eb0361b60fa None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:41.994298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fad234-3f3e-4314-ba71-2eb0361b60fa None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:42.106784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e439dc4f-8f99-4688-b54e-a5a6f57053ec admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:42.118408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde87a1-1e24-4295-8f4c-63f85a0fbc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:42.119049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde87a1-1e24-4295-8f4c-63f85a0fbc5a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.119328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde87a1-1e24-4295-8f4c-63f85a0fbc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.120529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde87a1-1e24-4295-8f4c-63f85a0fbc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.122461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde87a1-1e24-4295-8f4c-63f85a0fbc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:42.125043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbde87a1-1e24-4295-8f4c-63f85a0fbc5a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:42.138198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb07d1-4a9f-4d76-97b3-c2a932d533d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:42.138814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb07d1-4a9f-4d76-97b3-c2a932d533d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.139091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb07d1-4a9f-4d76-97b3-c2a932d533d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.139359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb07d1-4a9f-4d76-97b3-c2a932d533d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.139987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb07d1-4a9f-4d76-97b3-c2a932d533d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:42.141924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb07d1-4a9f-4d76-97b3-c2a932d533d0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:42.144391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 876/1741] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:41 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:42.148189 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde87a1-1e24-4295-8f4c-63f85a0fbc5a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:42.171420 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde87a1-1e24-4295-8f4c-63f85a0fbc5a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:42.177751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f34f4-836a-4c2b-aa9e-339cdbd7b618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:42.179102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f34f4-836a-4c2b-aa9e-339cdbd7b618 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.179570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f34f4-836a-4c2b-aa9e-339cdbd7b618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.179996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488f34f4-836a-4c2b-aa9e-339cdbd7b618 admin admin] PATH_INFO: `/v3/projects/0a2dd772b1624f23ab1716ca79e1e70a/users/b4294aaf1f1348c98c11f4f3a1323613/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.200044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f34f4-836a-4c2b-aa9e-339cdbd7b618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2dd772b1624f23ab1716ca79e1e70a, user_id=b4294aaf1f1348c98c11f4f3a1323613, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:42.201739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 866/1742] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:41 2022] POST /identity/v3/users => generated 306 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:42.203240 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488f34f4-836a-4c2b-aa9e-339cdbd7b618 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:42.206208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb07d1-4a9f-4d76-97b3-c2a932d533d0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:42.226537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb07d1-4a9f-4d76-97b3-c2a932d533d0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:42.233516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 877/1743] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:42 2022] PUT /identity/v3/projects/0a2dd772b1624f23ab1716ca79e1e70a/users/b4294aaf1f1348c98c11f4f3a1323613/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:42.237023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f34f4-836a-4c2b-aa9e-339cdbd7b618 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:42.332577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83fe91-99ac-4038-ae14-7765e524d012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:42.333731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83fe91-99ac-4038-ae14-7765e524d012 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.333869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83fe91-99ac-4038-ae14-7765e524d012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.334178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad83fe91-99ac-4038-ae14-7765e524d012 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.335250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83fe91-99ac-4038-ae14-7765e524d012 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:42.336969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad83fe91-99ac-4038-ae14-7765e524d012 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:42.337736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488f34f4-836a-4c2b-aa9e-339cdbd7b618 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:42.348257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd595fe-45c9-4dbf-9476-f06a7e239475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:42.349723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd595fe-45c9-4dbf-9476-f06a7e239475 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.350188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd595fe-45c9-4dbf-9476-f06a7e239475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.350956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd595fe-45c9-4dbf-9476-f06a7e239475 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.352098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd595fe-45c9-4dbf-9476-f06a7e239475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:42.354819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 867/1744] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:42.355269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd595fe-45c9-4dbf-9476-f06a7e239475 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:42.356147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83fe91-99ac-4038-ae14-7765e524d012 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:42.358206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ad83fe91-99ac-4038-ae14-7765e524d012 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:42.375490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad83fe91-99ac-4038-ae14-7765e524d012 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:42.376059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 878/1745] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:42.377087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd595fe-45c9-4dbf-9476-f06a7e239475 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:42.381200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:42.381838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.382245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.382534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] PATH_INFO: `/v3/projects/49bb165523c54eeb87c88e83a18f6750` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.383707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] RBAC: Authorizing `identity:delete_project(project_id=49bb165523c54eeb87c88e83a18f6750)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:42.387197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:42.393337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd595fe-45c9-4dbf-9476-f06a7e239475 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:42.399930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e72adc-cf36-4670-80ce-99223ddaa258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:42.400681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e72adc-cf36-4670-80ce-99223ddaa258 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.400995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e72adc-cf36-4670-80ce-99223ddaa258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.401302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e72adc-cf36-4670-80ce-99223ddaa258 admin admin] PATH_INFO: `/v3/projects/a7c5f821a7124b3994111ff6a24adf4a/users/d4c5935b1ead4f229f1a19fcc8959fa8/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.437652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e72adc-cf36-4670-80ce-99223ddaa258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c5f821a7124b3994111ff6a24adf4a, user_id=d4c5935b1ead4f229f1a19fcc8959fa8, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:42.442774 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e72adc-cf36-4670-80ce-99223ddaa258 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:42.469376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 879/1746] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:42 2022] PUT /identity/v3/projects/a7c5f821a7124b3994111ff6a24adf4a/users/d4c5935b1ead4f229f1a19fcc8959fa8/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:42.470612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e72adc-cf36-4670-80ce-99223ddaa258 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:42.528932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49bb165523c54eeb87c88e83a18f6750', 'request_id': 'req-c21763c6-9c6f-4638-8b57-69cb9566367a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:42.532425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:42.552700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:42.558556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49bb165523c54eeb87c88e83a18f6750', 'request_id': 'req-c21763c6-9c6f-4638-8b57-69cb9566367a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:42.572108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49bb165523c54eeb87c88e83a18f6750', 'request_id': 'req-c21763c6-9c6f-4638-8b57-69cb9566367a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:42.583255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e72adc-cf36-4670-80ce-99223ddaa258 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:42.596175 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] The token cache is being invalidate because project 49bb165523c54eeb87c88e83a18f6750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:42.596466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:42.597191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:42.603805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 868/1747] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:42 2022] DELETE /identity/v3/projects/49bb165523c54eeb87c88e83a18f6750 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:42.605575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:42.607056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:42.674131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ee957-7196-44f2-8fbb-365e6dcddd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:42.674903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ee957-7196-44f2-8fbb-365e6dcddd3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.675655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ee957-7196-44f2-8fbb-365e6dcddd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.675655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3ee957-7196-44f2-8fbb-365e6dcddd3c admin admin] PATH_INFO: `/v3/projects/0a2dd772b1624f23ab1716ca79e1e70a/users/b4294aaf1f1348c98c11f4f3a1323613/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.682223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ee957-7196-44f2-8fbb-365e6dcddd3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2dd772b1624f23ab1716ca79e1e70a, user_id=b4294aaf1f1348c98c11f4f3a1323613, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:42.686050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3ee957-7196-44f2-8fbb-365e6dcddd3c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:42.698142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21763c6-9c6f-4638-8b57-69cb9566367a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:42.701683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 880/1748] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:42 2022] PUT /identity/v3/projects/0a2dd772b1624f23ab1716ca79e1e70a/users/b4294aaf1f1348c98c11f4f3a1323613/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:42.702685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ee957-7196-44f2-8fbb-365e6dcddd3c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:42.714730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8f3ee957-7196-44f2-8fbb-365e6dcddd3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:42.787219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a4760d-55dd-41ca-9055-03315f920241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:42.788135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4760d-55dd-41ca-9055-03315f920241 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.788860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4760d-55dd-41ca-9055-03315f920241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.789066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4760d-55dd-41ca-9055-03315f920241 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.790112 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4760d-55dd-41ca-9055-03315f920241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:42.793590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a4760d-55dd-41ca-9055-03315f920241 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:42.805692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 869/1749] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:42 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:42.808149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b6e20-bdd8-49d1-92b6-7b60b090cfdd None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.808281 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b6e20-bdd8-49d1-92b6-7b60b090cfdd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.808505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59b6e20-bdd8-49d1-92b6-7b60b090cfdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.843796 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a59b6e20-bdd8-49d1-92b6-7b60b090cfdd None None] MFA Rules not processed for user `b4294aaf1f1348c98c11f4f3a1323613`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:42.845320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a59b6e20-bdd8-49d1-92b6-7b60b090cfdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:42.868569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3ee957-7196-44f2-8fbb-365e6dcddd3c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:42.874083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:42.875026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.876703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.877067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] PATH_INFO: `/v3/users/ba24046dd3f24d36b1f881881fbd7354` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.878901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba24046dd3f24d36b1f881881fbd7354)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:42.881555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:42.958328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba24046dd3f24d36b1f881881fbd7354'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:42.959426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba24046dd3f24d36b1f881881fbd7354'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:42.960086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba24046dd3f24d36b1f881881fbd7354'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:42.969264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 870/1750] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:42.971455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b6e20-bdd8-49d1-92b6-7b60b090cfdd None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:42.977456 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba24046dd3f24d36b1f881881fbd7354'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:42.984265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59b6e20-bdd8-49d1-92b6-7b60b090cfdd None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:42.985838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba24046dd3f24d36b1f881881fbd7354'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:42.990646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027aa553-8048-4ebc-a771-9f7a8aed3f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:42.991444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027aa553-8048-4ebc-a771-9f7a8aed3f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:42.991730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027aa553-8048-4ebc-a771-9f7a8aed3f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:42.992004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027aa553-8048-4ebc-a771-9f7a8aed3f17 admin admin] PATH_INFO: `/v3/projects/a7c5f821a7124b3994111ff6a24adf4a/users/d4c5935b1ead4f229f1a19fcc8959fa8/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:42.995255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027aa553-8048-4ebc-a771-9f7a8aed3f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7c5f821a7124b3994111ff6a24adf4a, user_id=d4c5935b1ead4f229f1a19fcc8959fa8, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:42.998698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027aa553-8048-4ebc-a771-9f7a8aed3f17 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:43.001184 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 881/1751] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:42 2022] DELETE /identity/v3/users/ba24046dd3f24d36b1f881881fbd7354 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:43.002098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:43.003391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:43.006722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:43.015130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:43.020275 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 871/1752] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:42 2022] PUT /identity/v3/projects/a7c5f821a7124b3994111ff6a24adf4a/users/d4c5935b1ead4f229f1a19fcc8959fa8/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:43.034030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26c159-beaa-413a-9084-4f7ad301abfc None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:43.034457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26c159-beaa-413a-9084-4f7ad301abfc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:43.034798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26c159-beaa-413a-9084-4f7ad301abfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:43.077167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3f26c159-beaa-413a-9084-4f7ad301abfc None None] MFA Rules not processed for user `d4c5935b1ead4f229f1a19fcc8959fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:43.079253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3f26c159-beaa-413a-9084-4f7ad301abfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:43.083423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe51d3-8e4c-4cff-b192-367bdd39b8d0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:43.146964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ba300-4ed1-46d9-be1a-a6e38feb765a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:43.147292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ba300-4ed1-46d9-be1a-a6e38feb765a service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:43.147566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ba300-4ed1-46d9-be1a-a6e38feb765a service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:43.148329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ba300-4ed1-46d9-be1a-a6e38feb765a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:43.150135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f27ba300-4ed1-46d9-be1a-a6e38feb765a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:43.190005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 872/1753] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:43 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:43.191254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26c159-beaa-413a-9084-4f7ad301abfc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:43.192710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3f26c159-beaa-413a-9084-4f7ad301abfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:43.207809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26c159-beaa-413a-9084-4f7ad301abfc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:43.214757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dff56-5c85-41d5-93b8-1e92877c988a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:43.215450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dff56-5c85-41d5-93b8-1e92877c988a service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:43.215731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dff56-5c85-41d5-93b8-1e92877c988a service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:43.216007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dff56-5c85-41d5-93b8-1e92877c988a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:43.224226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ba300-4ed1-46d9-be1a-a6e38feb765a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:43.225686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ba300-4ed1-46d9-be1a-a6e38feb765a service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:43.227371 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dff56-5c85-41d5-93b8-1e92877c988a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:43.230125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dff56-5c85-41d5-93b8-1e92877c988a service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:43.237095 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 882/1754] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:43.256610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 873/1755] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:43 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:43.367062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ba300-4ed1-46d9-be1a-a6e38feb765a service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:43.368581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f27ba300-4ed1-46d9-be1a-a6e38feb765a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:43.519107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ba300-4ed1-46d9-be1a-a6e38feb765a service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:43.532248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:43.533217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:43.533486 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:43.533741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] PATH_INFO: `/v3/projects/74f5099bd59845158c693ae33b4fc8ce` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:43.535660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74f5099bd59845158c693ae33b4fc8ce)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:43.537919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:43.607223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:43.643558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c7b5-d713-4b44-bdb6-ca3341af74ef None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:43.644122 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c7b5-d713-4b44-bdb6-ca3341af74ef None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:43.644656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927c7b5-d713-4b44-bdb6-ca3341af74ef None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:43.647081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 874/1756] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:10:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:10:43.658545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74f5099bd59845158c693ae33b4fc8ce', 'request_id': 'req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:43.669957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74f5099bd59845158c693ae33b4fc8ce', 'request_id': 'req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:43.677148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74f5099bd59845158c693ae33b4fc8ce', 'request_id': 'req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:43.716397 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] The token cache is being invalidate because project 74f5099bd59845158c693ae33b4fc8ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:43.716700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:43.717829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cc550-b515-4f9a-8482-173c171d31ea None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:43.718441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cc550-b515-4f9a-8482-173c171d31ea None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:43.718623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:43.719092 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26cc550-b515-4f9a-8482-173c171d31ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:43.724329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 883/1757] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:43 2022] DELETE /identity/v3/projects/74f5099bd59845158c693ae33b4fc8ce => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:43.725290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:43.731685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:43.755588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b26cc550-b515-4f9a-8482-173c171d31ea None None] MFA Rules not processed for user `43d6ba7b68884509a4e2859ceb72ef0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:43.757870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b26cc550-b515-4f9a-8482-173c171d31ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:43.895124 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb0bbf6-9692-48c4-9703-4e24a69d35e1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:43.905660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee8ad6-4a43-43c3-aa25-d1fb1e65715f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:43.906290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee8ad6-4a43-43c3-aa25-d1fb1e65715f service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:43.906567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee8ad6-4a43-43c3-aa25-d1fb1e65715f service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:43.906842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee8ad6-4a43-43c3-aa25-d1fb1e65715f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:43.910283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9eee8ad6-4a43-43c3-aa25-d1fb1e65715f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:43.925186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 875/1758] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:10:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:43.929284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00f603-3930-43fc-8386-93a6c2ccc197 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:43.929284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00f603-3930-43fc-8386-93a6c2ccc197 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:43.929284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00f603-3930-43fc-8386-93a6c2ccc197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:43.973692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9b00f603-3930-43fc-8386-93a6c2ccc197 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:43.974905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9b00f603-3930-43fc-8386-93a6c2ccc197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:44.047213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee8ad6-4a43-43c3-aa25-d1fb1e65715f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:44.048711 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee8ad6-4a43-43c3-aa25-d1fb1e65715f service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:44.076736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 884/1759] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:43 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:44.078422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee8ad6-4a43-43c3-aa25-d1fb1e65715f service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:44.082008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9eee8ad6-4a43-43c3-aa25-d1fb1e65715f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:44.113831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 876/1760] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:44.116173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f5fc6-93f9-4338-b731-47f7ea6ee391 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:44.116173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f5fc6-93f9-4338-b731-47f7ea6ee391 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:44.116412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0f5fc6-93f9-4338-b731-47f7ea6ee391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:44.119329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee8ad6-4a43-43c3-aa25-d1fb1e65715f service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:44.128783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc377ef-3bce-4342-908d-2faa0a6a3987 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:44.129906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc377ef-3bce-4342-908d-2faa0a6a3987 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:44.130336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc377ef-3bce-4342-908d-2faa0a6a3987 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:44.130757 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc377ef-3bce-4342-908d-2faa0a6a3987 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:44.133239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:44.148900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc377ef-3bce-4342-908d-2faa0a6a3987 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:44.151554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc377ef-3bce-4342-908d-2faa0a6a3987 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:44.158566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0d0f5fc6-93f9-4338-b731-47f7ea6ee391 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:44.159790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0d0f5fc6-93f9-4338-b731-47f7ea6ee391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:44.165570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 885/1761] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:44.207660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 877/1762] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:44.212441 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc377ef-3bce-4342-908d-2faa0a6a3987 service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:44.227427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc377ef-3bce-4342-908d-2faa0a6a3987 service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:44.237580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5347f-e76e-465c-a2a8-bcf850e354cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:44.238166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5347f-e76e-465c-a2a8-bcf850e354cd admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:44.238438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5347f-e76e-465c-a2a8-bcf850e354cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:44.238704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a5347f-e76e-465c-a2a8-bcf850e354cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:44.239352 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5347f-e76e-465c-a2a8-bcf850e354cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:44.239650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5347f-e76e-465c-a2a8-bcf850e354cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:44.249523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a5347f-e76e-465c-a2a8-bcf850e354cd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:44.270244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 886/1763] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:44.273426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f5fc6-93f9-4338-b731-47f7ea6ee391 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:44.297679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0f5fc6-93f9-4338-b731-47f7ea6ee391 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:44.302600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d76fb5-d8af-43be-ad95-1d18b9e7cfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:44.303215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d76fb5-d8af-43be-ad95-1d18b9e7cfca admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:44.303495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d76fb5-d8af-43be-ad95-1d18b9e7cfca admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:44.303767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d76fb5-d8af-43be-ad95-1d18b9e7cfca admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:44.304446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d76fb5-d8af-43be-ad95-1d18b9e7cfca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:44.306699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d76fb5-d8af-43be-ad95-1d18b9e7cfca admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:44.349154 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 878/1764] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:44 2022] POST /identity/v3/projects => generated 367 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:44.353059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5347f-e76e-465c-a2a8-bcf850e354cd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:44.369211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a5347f-e76e-465c-a2a8-bcf850e354cd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:44.408556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:44.450078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75d492-846a-451a-abcd-f92ab02bc391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:44.450803 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75d492-846a-451a-abcd-f92ab02bc391 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:44.451417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75d492-846a-451a-abcd-f92ab02bc391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:44.451417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75d492-846a-451a-abcd-f92ab02bc391 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:44.452161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75d492-846a-451a-abcd-f92ab02bc391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:44.454311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75d492-846a-451a-abcd-f92ab02bc391 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:44.485455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 887/1765] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:44 2022] POST /identity/v3/users => generated 284 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:44.486535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d76fb5-d8af-43be-ad95-1d18b9e7cfca admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:44.508300 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d76fb5-d8af-43be-ad95-1d18b9e7cfca admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:44.515036 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c223ad-f430-4424-ab85-0556ecf6beca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:44.515882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c223ad-f430-4424-ab85-0556ecf6beca admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:44.516318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c223ad-f430-4424-ab85-0556ecf6beca admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:44.516746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c223ad-f430-4424-ab85-0556ecf6beca admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:44.517647 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c223ad-f430-4424-ab85-0556ecf6beca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:44.519982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c223ad-f430-4424-ab85-0556ecf6beca admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:44.534628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 879/1766] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:44.538297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75d492-846a-451a-abcd-f92ab02bc391 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:44.551322 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75d492-846a-451a-abcd-f92ab02bc391 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:44.569201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c850012d-461e-4b4b-84b6-8133ff9b00a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:44.570252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850012d-461e-4b4b-84b6-8133ff9b00a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:44.570653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850012d-461e-4b4b-84b6-8133ff9b00a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:44.571604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c850012d-461e-4b4b-84b6-8133ff9b00a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:44.572574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850012d-461e-4b4b-84b6-8133ff9b00a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:44.574408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c850012d-461e-4b4b-84b6-8133ff9b00a7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:44.587605 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 888/1767] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:44.591825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c223ad-f430-4424-ab85-0556ecf6beca admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:44.602607 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c223ad-f430-4424-ab85-0556ecf6beca admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:44.609260 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8c9c7-393d-4956-a510-3073e636b833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:44.609921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c9c7-393d-4956-a510-3073e636b833 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:44.610199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c9c7-393d-4956-a510-3073e636b833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:44.610474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c8c9c7-393d-4956-a510-3073e636b833 admin admin] PATH_INFO: `/v3/projects/a33746b410c742d9b46e4e930a2b32ed/users/51f64119206f48c1805e8398c76c9612/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:44.628773 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c9c7-393d-4956-a510-3073e636b833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33746b410c742d9b46e4e930a2b32ed, user_id=51f64119206f48c1805e8398c76c9612, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:44.632139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c8c9c7-393d-4956-a510-3073e636b833 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:44.649191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 880/1768] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:44 2022] PUT /identity/v3/projects/a33746b410c742d9b46e4e930a2b32ed/users/51f64119206f48c1805e8398c76c9612/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:44.660994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e9093-b78f-45d2-9191-2019a40b2d65 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:44.661536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e9093-b78f-45d2-9191-2019a40b2d65 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:44.661536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e9093-b78f-45d2-9191-2019a40b2d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:44.690739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ca5e9093-b78f-45d2-9191-2019a40b2d65 None None] MFA Rules not processed for user `51f64119206f48c1805e8398c76c9612`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:44.691949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ca5e9093-b78f-45d2-9191-2019a40b2d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:44.758424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 889/1769] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:44.762679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8c9c7-393d-4956-a510-3073e636b833 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:44.831169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c8c9c7-393d-4956-a510-3073e636b833 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:44.846424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b7f61-ceb2-48dc-9e1f-4f28187c2b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:44.847276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b7f61-ceb2-48dc-9e1f-4f28187c2b27 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:44.847456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b7f61-ceb2-48dc-9e1f-4f28187c2b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:44.847720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146b7f61-ceb2-48dc-9e1f-4f28187c2b27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:44.848359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b7f61-ceb2-48dc-9e1f-4f28187c2b27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:44.850365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146b7f61-ceb2-48dc-9e1f-4f28187c2b27 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:44.880356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 881/1770] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:44 2022] POST /identity/v3/projects => generated 367 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:10:44.884692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e9093-b78f-45d2-9191-2019a40b2d65 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:44.898062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e9093-b78f-45d2-9191-2019a40b2d65 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:44.978954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081ee20-75d9-489e-9320-f0a6dfefd359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:44.980183 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081ee20-75d9-489e-9320-f0a6dfefd359 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:44.981731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081ee20-75d9-489e-9320-f0a6dfefd359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:44.982110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081ee20-75d9-489e-9320-f0a6dfefd359 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:44.983067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081ee20-75d9-489e-9320-f0a6dfefd359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:44.987982 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b081ee20-75d9-489e-9320-f0a6dfefd359 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:45.038088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 890/1771] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:44 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:10:45.042027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b7f61-ceb2-48dc-9e1f-4f28187c2b27 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:45.055329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146b7f61-ceb2-48dc-9e1f-4f28187c2b27 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:45.062712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afda0e0-11b1-4188-898a-e12c23da7bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:45.063483 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda0e0-11b1-4188-898a-e12c23da7bdf admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:45.063863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda0e0-11b1-4188-898a-e12c23da7bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:45.064259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afda0e0-11b1-4188-898a-e12c23da7bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:45.065059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afda0e0-11b1-4188-898a-e12c23da7bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:45.066841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afda0e0-11b1-4188-898a-e12c23da7bdf admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:45.080420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 882/1772] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:45.085078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081ee20-75d9-489e-9320-f0a6dfefd359 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:45.102727 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b081ee20-75d9-489e-9320-f0a6dfefd359 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:45.109917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bbb12-12fe-47d2-b095-6756f7779b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:45.110247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bbb12-12fe-47d2-b095-6756f7779b1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:45.110489 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bbb12-12fe-47d2-b095-6756f7779b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:45.110880 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38bbb12-12fe-47d2-b095-6756f7779b1e admin admin] PATH_INFO: `/v3/projects/0df84370946e444789407b8ac0e76e9c/users/6070d5ea9dcc478f8cf81d82b4aff4e5/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:45.134035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bbb12-12fe-47d2-b095-6756f7779b1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df84370946e444789407b8ac0e76e9c, user_id=6070d5ea9dcc478f8cf81d82b4aff4e5, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:45.140717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38bbb12-12fe-47d2-b095-6756f7779b1e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:45.215675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 891/1773] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:45 2022] PUT /identity/v3/projects/0df84370946e444789407b8ac0e76e9c/users/6070d5ea9dcc478f8cf81d82b4aff4e5/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:10:45.219173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afda0e0-11b1-4188-898a-e12c23da7bdf admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:45.242692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afda0e0-11b1-4188-898a-e12c23da7bdf admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:45.349220 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5461c4-9c9d-4c05-b4ff-2447964fd6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:45.350181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5461c4-9c9d-4c05-b4ff-2447964fd6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:45.350620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5461c4-9c9d-4c05-b4ff-2447964fd6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:45.351197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5461c4-9c9d-4c05-b4ff-2447964fd6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:45.352809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5461c4-9c9d-4c05-b4ff-2447964fd6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:45.354705 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5461c4-9c9d-4c05-b4ff-2447964fd6a3 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:45.374666 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 883/1774] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:45 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:45.378761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bbb12-12fe-47d2-b095-6756f7779b1e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:45.392865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38bbb12-12fe-47d2-b095-6756f7779b1e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:45.398167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614db39-2f28-4cb9-9fa8-4d5cd73a203b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:45.398972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614db39-2f28-4cb9-9fa8-4d5cd73a203b admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:45.399349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614db39-2f28-4cb9-9fa8-4d5cd73a203b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:45.399718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614db39-2f28-4cb9-9fa8-4d5cd73a203b admin admin] PATH_INFO: `/v3/projects/0df84370946e444789407b8ac0e76e9c/users/6070d5ea9dcc478f8cf81d82b4aff4e5/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:45.401872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614db39-2f28-4cb9-9fa8-4d5cd73a203b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df84370946e444789407b8ac0e76e9c, user_id=6070d5ea9dcc478f8cf81d82b4aff4e5, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:45.404840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3614db39-2f28-4cb9-9fa8-4d5cd73a203b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:45.408491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:45.421342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f4ae3-e724-46aa-906c-ffbedd682f35 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:45.421790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f4ae3-e724-46aa-906c-ffbedd682f35 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:45.422268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702f4ae3-e724-46aa-906c-ffbedd682f35 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:45.423920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 884/1775] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:10:45.433288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 892/1776] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:45 2022] PUT /identity/v3/projects/0df84370946e444789407b8ac0e76e9c/users/6070d5ea9dcc478f8cf81d82b4aff4e5/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:45.437378 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30473440-ad17-4ba0-8cf2-ba11646010e9 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:45.437664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30473440-ad17-4ba0-8cf2-ba11646010e9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:45.437929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30473440-ad17-4ba0-8cf2-ba11646010e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:45.454827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1e2d1-18eb-460d-b1e5-61f308590ffb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:45.455349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1e2d1-18eb-460d-b1e5-61f308590ffb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:45.456944 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1e2d1-18eb-460d-b1e5-61f308590ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:45.474221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:45.498161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e3e1e2d1-18eb-460d-b1e5-61f308590ffb None None] MFA Rules not processed for user `43d6ba7b68884509a4e2859ceb72ef0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:45.499808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e3e1e2d1-18eb-460d-b1e5-61f308590ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:45.501004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-30473440-ad17-4ba0-8cf2-ba11646010e9 None None] MFA Rules not processed for user `6070d5ea9dcc478f8cf81d82b4aff4e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:45.502206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-30473440-ad17-4ba0-8cf2-ba11646010e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:45.595958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 893/1777] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:10:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:45.596628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1e2d1-18eb-460d-b1e5-61f308590ffb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:45.598426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e3e1e2d1-18eb-460d-b1e5-61f308590ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:45.612985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 885/1778] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:45.612985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30473440-ad17-4ba0-8cf2-ba11646010e9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:45.634190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30473440-ad17-4ba0-8cf2-ba11646010e9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:45.643138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112af6b-e31b-48b3-b6d7-1091058a7dba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:45.646391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112af6b-e31b-48b3-b6d7-1091058a7dba service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:45.647009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112af6b-e31b-48b3-b6d7-1091058a7dba service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:45.647798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112af6b-e31b-48b3-b6d7-1091058a7dba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:45.660135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112af6b-e31b-48b3-b6d7-1091058a7dba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:45.662805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112af6b-e31b-48b3-b6d7-1091058a7dba service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:45.677862 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 886/1779] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:45.679506 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112af6b-e31b-48b3-b6d7-1091058a7dba service placement] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:45.706803 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e1e2d1-18eb-460d-b1e5-61f308590ffb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:45.738571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:45.740150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:45.740431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:45.740775 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] PATH_INFO: `/v3/users/89aad62ac3e34cb7a91403757a8c1b52` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:45.742050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] RBAC: Authorizing `identity:delete_user(user_id=89aad62ac3e34cb7a91403757a8c1b52)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:45.744376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:45.779143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112af6b-e31b-48b3-b6d7-1091058a7dba service placement] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:45.795602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1fc97-e70d-4fad-b417-5c72f72aa062 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:45.796935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1fc97-e70d-4fad-b417-5c72f72aa062 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:45.797465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1fc97-e70d-4fad-b417-5c72f72aa062 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:45.797749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c1fc97-e70d-4fad-b417-5c72f72aa062 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:45.822545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1fc97-e70d-4fad-b417-5c72f72aa062 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:45.827024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c1fc97-e70d-4fad-b417-5c72f72aa062 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:45.849876 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 887/1780] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:45.869007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89aad62ac3e34cb7a91403757a8c1b52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:45.872432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89aad62ac3e34cb7a91403757a8c1b52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:45.873121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89aad62ac3e34cb7a91403757a8c1b52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:45.896010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89aad62ac3e34cb7a91403757a8c1b52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:45.907319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89aad62ac3e34cb7a91403757a8c1b52'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:45.940823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1fc97-e70d-4fad-b417-5c72f72aa062 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:45.945589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 894/1781] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:45 2022] DELETE /identity/v3/users/89aad62ac3e34cb7a91403757a8c1b52 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:45.955667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c1fc97-e70d-4fad-b417-5c72f72aa062 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:46.020586 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a8670-fb9d-4f66-8a8f-de6c0a66783e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:46.021020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a8670-fb9d-4f66-8a8f-de6c0a66783e service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:46.021414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a8670-fb9d-4f66-8a8f-de6c0a66783e service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:46.021939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477a8670-fb9d-4f66-8a8f-de6c0a66783e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:46.024298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-477a8670-fb9d-4f66-8a8f-de6c0a66783e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:46.085130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:46.113139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a8670-fb9d-4f66-8a8f-de6c0a66783e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:46.114609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477a8670-fb9d-4f66-8a8f-de6c0a66783e service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:46.140337 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 888/1782] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:10:46.194511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:46.195996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:46.217466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:46.307662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88133243-e338-440f-bebb-cc9ec0a1c86f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:46.316055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:46.316707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:46.316990 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:46.317305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] PATH_INFO: `/v3/projects/a54c1053450d417d833aaf8e73fe7901` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:46.319435 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a54c1053450d417d833aaf8e73fe7901)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:46.321537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:46.438007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a54c1053450d417d833aaf8e73fe7901', 'request_id': 'req-2a8258c5-ab04-40da-ac28-f1319c99f560'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:46.446960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a54c1053450d417d833aaf8e73fe7901', 'request_id': 'req-2a8258c5-ab04-40da-ac28-f1319c99f560'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:46.456626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a54c1053450d417d833aaf8e73fe7901', 'request_id': 'req-2a8258c5-ab04-40da-ac28-f1319c99f560'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:46.493031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] The token cache is being invalidate because project a54c1053450d417d833aaf8e73fe7901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:46.493379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:46.494078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2a8258c5-ab04-40da-ac28-f1319c99f560 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:46.505006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 895/1783] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:46 2022] DELETE /identity/v3/projects/a54c1053450d417d833aaf8e73fe7901 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:46.509225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a8670-fb9d-4f66-8a8f-de6c0a66783e service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:46.525468 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477a8670-fb9d-4f66-8a8f-de6c0a66783e service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:46.539704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83736636-0340-4535-a360-5554ea4d7a93 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:46.539989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83736636-0340-4535-a360-5554ea4d7a93 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:46.540350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83736636-0340-4535-a360-5554ea4d7a93 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:46.542172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 896/1784] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:10:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:10:46.576106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468ca4-13d7-4bb7-8acf-aa84f9f1568b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:46.577163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468ca4-13d7-4bb7-8acf-aa84f9f1568b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:46.577631 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96468ca4-13d7-4bb7-8acf-aa84f9f1568b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:46.634021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-96468ca4-13d7-4bb7-8acf-aa84f9f1568b None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:46.638284 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-96468ca4-13d7-4bb7-8acf-aa84f9f1568b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:46.648568 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:46.649651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:46.650242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:46.650676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] PATH_INFO: `/v3/users/90153a04413b4e3bb68e50a591b62abd` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:46.653006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90153a04413b4e3bb68e50a591b62abd)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:46.655223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:46.731673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90153a04413b4e3bb68e50a591b62abd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:46.732589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90153a04413b4e3bb68e50a591b62abd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:46.740258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90153a04413b4e3bb68e50a591b62abd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:46.740926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90153a04413b4e3bb68e50a591b62abd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:46.755222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90153a04413b4e3bb68e50a591b62abd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:46.756659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 897/1785] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:10:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:46.779964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 889/1786] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:46 2022] DELETE /identity/v3/users/90153a04413b4e3bb68e50a591b62abd => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:46.782279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96468ca4-13d7-4bb7-8acf-aa84f9f1568b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:46.784072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-96468ca4-13d7-4bb7-8acf-aa84f9f1568b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:46.798924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96468ca4-13d7-4bb7-8acf-aa84f9f1568b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:46.831602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:46.876072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101c7b86-2700-4f5a-b8e9-e102fbeafc2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:46.877639 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7b86-2700-4f5a-b8e9-e102fbeafc2f service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:46.877995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7b86-2700-4f5a-b8e9-e102fbeafc2f service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:46.879834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c7b86-2700-4f5a-b8e9-e102fbeafc2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:46.894885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c7b86-2700-4f5a-b8e9-e102fbeafc2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:46.898906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101c7b86-2700-4f5a-b8e9-e102fbeafc2f service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:46.913595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 898/1787] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:47.005331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:47.006751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:47.092649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaebeca-1456-4ee3-9d26-fb44ec3f54a7 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:47.097404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:47.098031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.098310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.098583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] PATH_INFO: `/v3/projects/7ec968a39acf4c4ab15291c604002055` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.099741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ec968a39acf4c4ab15291c604002055)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:47.101717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:47.103438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:47.155857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ec968a39acf4c4ab15291c604002055', 'request_id': 'req-6ba239c3-cd78-4946-8ca8-b528de5adf6a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:47.164763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ec968a39acf4c4ab15291c604002055', 'request_id': 'req-6ba239c3-cd78-4946-8ca8-b528de5adf6a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:47.172306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ec968a39acf4c4ab15291c604002055', 'request_id': 'req-6ba239c3-cd78-4946-8ca8-b528de5adf6a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:47.182955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] The token cache is being invalidate because project 7ec968a39acf4c4ab15291c604002055 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:47.183244 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:47.183888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6ba239c3-cd78-4946-8ca8-b528de5adf6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:47.185410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 890/1788] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:47 2022] DELETE /identity/v3/projects/7ec968a39acf4c4ab15291c604002055 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:47.190572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d58fe6-b80c-4f57-ab21-a46f91ddbed7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.190863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d58fe6-b80c-4f57-ab21-a46f91ddbed7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.191141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d58fe6-b80c-4f57-ab21-a46f91ddbed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.210569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-00d58fe6-b80c-4f57-ab21-a46f91ddbed7 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:47.211768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-00d58fe6-b80c-4f57-ab21-a46f91ddbed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:47.272051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 899/1789] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:47.280438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bed8d-4ff7-4683-b9c3-cb627ddd5cb0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.280722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bed8d-4ff7-4683-b9c3-cb627ddd5cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.280994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bed8d-4ff7-4683-b9c3-cb627ddd5cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.300071 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0e7bed8d-4ff7-4683-b9c3-cb627ddd5cb0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:47.301307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0e7bed8d-4ff7-4683-b9c3-cb627ddd5cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:47.309337 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 891/1790] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:47.313543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d58fe6-b80c-4f57-ab21-a46f91ddbed7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:47.323464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d58fe6-b80c-4f57-ab21-a46f91ddbed7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:47.327994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442693cb-8f5f-4990-a84e-4659fcfc8e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:47.328590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442693cb-8f5f-4990-a84e-4659fcfc8e84 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.328869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442693cb-8f5f-4990-a84e-4659fcfc8e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.329166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442693cb-8f5f-4990-a84e-4659fcfc8e84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.329802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442693cb-8f5f-4990-a84e-4659fcfc8e84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:47.330103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442693cb-8f5f-4990-a84e-4659fcfc8e84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:47.331668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442693cb-8f5f-4990-a84e-4659fcfc8e84 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:47.343520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 900/1791] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:47.347647 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bed8d-4ff7-4683-b9c3-cb627ddd5cb0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:47.356992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bed8d-4ff7-4683-b9c3-cb627ddd5cb0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:47.361608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fccac-ebbb-4dbc-b63c-796a1ee701fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:47.362199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fccac-ebbb-4dbc-b63c-796a1ee701fa admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.362505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fccac-ebbb-4dbc-b63c-796a1ee701fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.362774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fccac-ebbb-4dbc-b63c-796a1ee701fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.363394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fccac-ebbb-4dbc-b63c-796a1ee701fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:47.365300 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fccac-ebbb-4dbc-b63c-796a1ee701fa admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:47.395068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 892/1792] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:47 2022] POST /identity/v3/projects => generated 367 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:47.396154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442693cb-8f5f-4990-a84e-4659fcfc8e84 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:47.407045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442693cb-8f5f-4990-a84e-4659fcfc8e84 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:47.472709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b9eb2-24e9-4f35-8bc6-f345c1ab2c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:47.473359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b9eb2-24e9-4f35-8bc6-f345c1ab2c97 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.473637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b9eb2-24e9-4f35-8bc6-f345c1ab2c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.474334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68b9eb2-24e9-4f35-8bc6-f345c1ab2c97 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.474557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b9eb2-24e9-4f35-8bc6-f345c1ab2c97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:47.476455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68b9eb2-24e9-4f35-8bc6-f345c1ab2c97 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:47.502296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 901/1793] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:47 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:10:47.503490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fccac-ebbb-4dbc-b63c-796a1ee701fa admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:47.518256 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fccac-ebbb-4dbc-b63c-796a1ee701fa admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:47.523639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4abfa-b1c3-4ce8-b36b-945855c86f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:47.524186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4abfa-b1c3-4ce8-b36b-945855c86f1a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.524461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4abfa-b1c3-4ce8-b36b-945855c86f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.524730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4abfa-b1c3-4ce8-b36b-945855c86f1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.525415 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4abfa-b1c3-4ce8-b36b-945855c86f1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:47.527003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4abfa-b1c3-4ce8-b36b-945855c86f1a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:47.541700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 893/1794] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:47.543001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b9eb2-24e9-4f35-8bc6-f345c1ab2c97 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:47.553459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68b9eb2-24e9-4f35-8bc6-f345c1ab2c97 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:47.559564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfaf090-b878-4027-9602-1591854e0d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:47.560281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf090-b878-4027-9602-1591854e0d6c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.560571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf090-b878-4027-9602-1591854e0d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.560841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfaf090-b878-4027-9602-1591854e0d6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.561587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaf090-b878-4027-9602-1591854e0d6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:47.563169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfaf090-b878-4027-9602-1591854e0d6c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:47.574645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 902/1795] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:47.578827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4abfa-b1c3-4ce8-b36b-945855c86f1a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:47.589092 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4abfa-b1c3-4ce8-b36b-945855c86f1a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:47.593761 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23783d27-4eca-4aa3-952d-93a932c48b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:47.594463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23783d27-4eca-4aa3-952d-93a932c48b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.594742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23783d27-4eca-4aa3-952d-93a932c48b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.595010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23783d27-4eca-4aa3-952d-93a932c48b9c admin admin] PATH_INFO: `/v3/projects/be02d47a320647c9be5d68df2a1d941d/users/81d5a957d35d4a84ba892a38f5bb100f/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.610103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23783d27-4eca-4aa3-952d-93a932c48b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=be02d47a320647c9be5d68df2a1d941d, user_id=81d5a957d35d4a84ba892a38f5bb100f, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:47.612702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23783d27-4eca-4aa3-952d-93a932c48b9c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:47.625744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 894/1796] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:47 2022] PUT /identity/v3/projects/be02d47a320647c9be5d68df2a1d941d/users/81d5a957d35d4a84ba892a38f5bb100f/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:10:47.635965 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164865ef-a134-4a83-a598-36b899487336 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.636320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164865ef-a134-4a83-a598-36b899487336 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.636739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164865ef-a134-4a83-a598-36b899487336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.665509 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-164865ef-a134-4a83-a598-36b899487336 None None] MFA Rules not processed for user `81d5a957d35d4a84ba892a38f5bb100f`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:47.667723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-164865ef-a134-4a83-a598-36b899487336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:47.683595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:47.740612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 903/1797] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:47.745001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23783d27-4eca-4aa3-952d-93a932c48b9c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:47.815519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23783d27-4eca-4aa3-952d-93a932c48b9c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:47.820517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c0ffa-8ca1-47e3-a725-dc318b6397ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:47.821092 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c0ffa-8ca1-47e3-a725-dc318b6397ce admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.821399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c0ffa-8ca1-47e3-a725-dc318b6397ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.821717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c0ffa-8ca1-47e3-a725-dc318b6397ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.822776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c0ffa-8ca1-47e3-a725-dc318b6397ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:47.825174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c0ffa-8ca1-47e3-a725-dc318b6397ce admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:47.857793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 895/1798] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:47 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:47.862114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164865ef-a134-4a83-a598-36b899487336 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:47.873229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164865ef-a134-4a83-a598-36b899487336 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:47.930613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164e0ec-4f26-4910-bae4-dd779007db96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:47.931487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164e0ec-4f26-4910-bae4-dd779007db96 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.931804 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164e0ec-4f26-4910-bae4-dd779007db96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.932050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6164e0ec-4f26-4910-bae4-dd779007db96 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.932680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164e0ec-4f26-4910-bae4-dd779007db96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:47.934601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6164e0ec-4f26-4910-bae4-dd779007db96 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:47.957484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 904/1799] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:47 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:47.961620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c0ffa-8ca1-47e3-a725-dc318b6397ce admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:47.972748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c0ffa-8ca1-47e3-a725-dc318b6397ce admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:47.977517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e854c1-57a3-4227-818a-cd3b37c1f542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:47.978114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e854c1-57a3-4227-818a-cd3b37c1f542 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:47.978416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e854c1-57a3-4227-818a-cd3b37c1f542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:47.979395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e854c1-57a3-4227-818a-cd3b37c1f542 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:47.980051 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e854c1-57a3-4227-818a-cd3b37c1f542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:47.981676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e854c1-57a3-4227-818a-cd3b37c1f542 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:47.988201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:47.997699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 896/1800] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:47.999220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164e0ec-4f26-4910-bae4-dd779007db96 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:48.011854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6164e0ec-4f26-4910-bae4-dd779007db96 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:48.017464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02a74a-ca29-4b1f-ab32-00ea3b261ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:48.018120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02a74a-ca29-4b1f-ab32-00ea3b261ade admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:48.018400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02a74a-ca29-4b1f-ab32-00ea3b261ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:48.018675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02a74a-ca29-4b1f-ab32-00ea3b261ade admin admin] PATH_INFO: `/v3/projects/ed0c44c97e7d4759ab6e2af00b25b316/users/8798ed52ba4a4ef191841c203c9d38ea/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:48.035870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02a74a-ca29-4b1f-ab32-00ea3b261ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0c44c97e7d4759ab6e2af00b25b316, user_id=8798ed52ba4a4ef191841c203c9d38ea, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:48.039390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02a74a-ca29-4b1f-ab32-00ea3b261ade admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:48.067172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 905/1801] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:47 2022] PUT /identity/v3/projects/ed0c44c97e7d4759ab6e2af00b25b316/users/8798ed52ba4a4ef191841c203c9d38ea/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:48.068621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e854c1-57a3-4227-818a-cd3b37c1f542 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:48.080387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e854c1-57a3-4227-818a-cd3b37c1f542 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:48.151459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c357f2-1f1e-4cb1-aa1f-9bec9ada1007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:48.152216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c357f2-1f1e-4cb1-aa1f-9bec9ada1007 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:48.152327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c357f2-1f1e-4cb1-aa1f-9bec9ada1007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:48.152585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c357f2-1f1e-4cb1-aa1f-9bec9ada1007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:48.153263 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c357f2-1f1e-4cb1-aa1f-9bec9ada1007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:48.154864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c357f2-1f1e-4cb1-aa1f-9bec9ada1007 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:48.164812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 897/1802] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:48 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:48.170949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02a74a-ca29-4b1f-ab32-00ea3b261ade admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:48.183252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02a74a-ca29-4b1f-ab32-00ea3b261ade admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:48.191331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37adae77-7249-4f7a-ada5-1bdeba61a272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:48.192186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adae77-7249-4f7a-ada5-1bdeba61a272 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:48.192579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adae77-7249-4f7a-ada5-1bdeba61a272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:48.192930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37adae77-7249-4f7a-ada5-1bdeba61a272 admin admin] PATH_INFO: `/v3/projects/ed0c44c97e7d4759ab6e2af00b25b316/users/8798ed52ba4a4ef191841c203c9d38ea/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:48.195263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37adae77-7249-4f7a-ada5-1bdeba61a272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0c44c97e7d4759ab6e2af00b25b316, user_id=8798ed52ba4a4ef191841c203c9d38ea, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:48.199660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37adae77-7249-4f7a-ada5-1bdeba61a272 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:48.213001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 906/1803] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:48 2022] PUT /identity/v3/projects/ed0c44c97e7d4759ab6e2af00b25b316/users/8798ed52ba4a4ef191841c203c9d38ea/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:48.224533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51aeec6-08a0-4384-929f-3d74c07823f3 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:48.224849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51aeec6-08a0-4384-929f-3d74c07823f3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:48.225091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51aeec6-08a0-4384-929f-3d74c07823f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:48.257591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c51aeec6-08a0-4384-929f-3d74c07823f3 None None] MFA Rules not processed for user `8798ed52ba4a4ef191841c203c9d38ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:48.257591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c51aeec6-08a0-4384-929f-3d74c07823f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:48.335360 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 898/1804] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:48.344271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37adae77-7249-4f7a-ada5-1bdeba61a272 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:48.354501 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-37adae77-7249-4f7a-ada5-1bdeba61a272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:48.415278 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37adae77-7249-4f7a-ada5-1bdeba61a272 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:48.419579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cac861-2184-413d-8927-84d8eb1a483b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:48.420186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cac861-2184-413d-8927-84d8eb1a483b service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:48.420467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cac861-2184-413d-8927-84d8eb1a483b service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:48.420738 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cac861-2184-413d-8927-84d8eb1a483b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:48.429728 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cac861-2184-413d-8927-84d8eb1a483b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:48.431400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cac861-2184-413d-8927-84d8eb1a483b service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:48.441987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 907/1805] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:49.919679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51aeec6-08a0-4384-929f-3d74c07823f3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:49.921844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c51aeec6-08a0-4384-929f-3d74c07823f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:50.045199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51aeec6-08a0-4384-929f-3d74c07823f3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:50.049159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cac861-2184-413d-8927-84d8eb1a483b service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:50.050597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:50.051237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:50.051803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:50.052304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] PATH_INFO: `/v3/users/51f64119206f48c1805e8398c76c9612` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:50.053657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51f64119206f48c1805e8398c76c9612)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:50.055757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:50.061966 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cac861-2184-413d-8927-84d8eb1a483b service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:50.069084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11dbb69-6ea7-4a28-8273-10880b5ec158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:50.069981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dbb69-6ea7-4a28-8273-10880b5ec158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:50.070790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dbb69-6ea7-4a28-8273-10880b5ec158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:50.071172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11dbb69-6ea7-4a28-8273-10880b5ec158 admin admin] PATH_INFO: `/v3/users/81d5a957d35d4a84ba892a38f5bb100f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:50.073808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11dbb69-6ea7-4a28-8273-10880b5ec158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d5a957d35d4a84ba892a38f5bb100f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:50.076527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11dbb69-6ea7-4a28-8273-10880b5ec158 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:50.079846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:50.085314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:50.113475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51f64119206f48c1805e8398c76c9612'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.114294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51f64119206f48c1805e8398c76c9612'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.125181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51f64119206f48c1805e8398c76c9612'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.126476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51f64119206f48c1805e8398c76c9612'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.138910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51f64119206f48c1805e8398c76c9612'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.142305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a11dbb69-6ea7-4a28-8273-10880b5ec158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d5a957d35d4a84ba892a38f5bb100f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.143144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a11dbb69-6ea7-4a28-8273-10880b5ec158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d5a957d35d4a84ba892a38f5bb100f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.144152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a11dbb69-6ea7-4a28-8273-10880b5ec158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d5a957d35d4a84ba892a38f5bb100f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.159224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 899/1806] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:49 2022] DELETE /identity/v3/users/51f64119206f48c1805e8398c76c9612 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:50.162345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:50.162461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a11dbb69-6ea7-4a28-8273-10880b5ec158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d5a957d35d4a84ba892a38f5bb100f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.162559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:50.172187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a11dbb69-6ea7-4a28-8273-10880b5ec158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d5a957d35d4a84ba892a38f5bb100f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.189943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 908/1807] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:50 2022] DELETE /identity/v3/users/81d5a957d35d4a84ba892a38f5bb100f => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:10:50.191661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e1a9b-3459-4d25-b8b0-f6cd06d6c94e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:50.191973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e1a9b-3459-4d25-b8b0-f6cd06d6c94e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:50.192251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e1a9b-3459-4d25-b8b0-f6cd06d6c94e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:50.212251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-714e1a9b-3459-4d25-b8b0-f6cd06d6c94e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:50.213800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-714e1a9b-3459-4d25-b8b0-f6cd06d6c94e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:50.235428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d708b94-e5b2-4d74-ac8e-f1d5cdd72ed2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:50.275551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 909/1808] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:50.279104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f82fb-4ed7-48a2-b9f7-3968f3208820 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:50.279104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f82fb-4ed7-48a2-b9f7-3968f3208820 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:50.279104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f82fb-4ed7-48a2-b9f7-3968f3208820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:50.296917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-116f82fb-4ed7-48a2-b9f7-3968f3208820 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:50.297277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4217ac52-23c6-4687-b598-8711e8113d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:50.297903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217ac52-23c6-4687-b598-8711e8113d08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:50.298157 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-116f82fb-4ed7-48a2-b9f7-3968f3208820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:50.298259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217ac52-23c6-4687-b598-8711e8113d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:50.298445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4217ac52-23c6-4687-b598-8711e8113d08 admin admin] PATH_INFO: `/v3/users/af7fb86fba9b49c3891894dda31e8e89` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:50.299605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217ac52-23c6-4687-b598-8711e8113d08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af7fb86fba9b49c3891894dda31e8e89)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:50.301608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4217ac52-23c6-4687-b598-8711e8113d08 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:50.305783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 910/1809] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:50.310188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f82fb-4ed7-48a2-b9f7-3968f3208820 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:50.310188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-116f82fb-4ed7-48a2-b9f7-3968f3208820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:50.356566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4217ac52-23c6-4687-b598-8711e8113d08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af7fb86fba9b49c3891894dda31e8e89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.357439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4217ac52-23c6-4687-b598-8711e8113d08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af7fb86fba9b49c3891894dda31e8e89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.359521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:50.367424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4217ac52-23c6-4687-b598-8711e8113d08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af7fb86fba9b49c3891894dda31e8e89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.368852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4217ac52-23c6-4687-b598-8711e8113d08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af7fb86fba9b49c3891894dda31e8e89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.376994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f82fb-4ed7-48a2-b9f7-3968f3208820 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:50.381066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf1c07-5435-45e2-adc4-fa68a821aef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:50.381991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf1c07-5435-45e2-adc4-fa68a821aef6 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:50.382280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf1c07-5435-45e2-adc4-fa68a821aef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:50.382558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf1c07-5435-45e2-adc4-fa68a821aef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:50.384334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f8cf1c07-5435-45e2-adc4-fa68a821aef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:50.393081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4217ac52-23c6-4687-b598-8711e8113d08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af7fb86fba9b49c3891894dda31e8e89'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:50.398069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf1c07-5435-45e2-adc4-fa68a821aef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:50.399843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf1c07-5435-45e2-adc4-fa68a821aef6 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:50.411778 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 911/1810] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:50.412623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 900/1811] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:50 2022] DELETE /identity/v3/users/af7fb86fba9b49c3891894dda31e8e89 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:50.415570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf1c07-5435-45e2-adc4-fa68a821aef6 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:50.417977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f8cf1c07-5435-45e2-adc4-fa68a821aef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:50.418987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9328746-c8a3-4444-b465-e55664b3e07f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:50.419781 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9328746-c8a3-4444-b465-e55664b3e07f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:50.420355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9328746-c8a3-4444-b465-e55664b3e07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:50.429668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf1c07-5435-45e2-adc4-fa68a821aef6 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:50.446536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:50.483911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d9328746-c8a3-4444-b465-e55664b3e07f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:50.488169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d9328746-c8a3-4444-b465-e55664b3e07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:50.500832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193553d4-6f75-4ce7-8cf5-d5db3e7afac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:50.501552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193553d4-6f75-4ce7-8cf5-d5db3e7afac6 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:50.501843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193553d4-6f75-4ce7-8cf5-d5db3e7afac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:50.502121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193553d4-6f75-4ce7-8cf5-d5db3e7afac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:50.504148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-193553d4-6f75-4ce7-8cf5-d5db3e7afac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:50.580207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 901/1812] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:50.592349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9328746-c8a3-4444-b465-e55664b3e07f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:50.594484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d9328746-c8a3-4444-b465-e55664b3e07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:50.599563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193553d4-6f75-4ce7-8cf5-d5db3e7afac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:50.601281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193553d4-6f75-4ce7-8cf5-d5db3e7afac6 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:50.613893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 912/1813] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:50.658371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193553d4-6f75-4ce7-8cf5-d5db3e7afac6 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:50.670407 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193553d4-6f75-4ce7-8cf5-d5db3e7afac6 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:50.680083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c56edb-bd9f-428c-8c17-c8c667940050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:50.680611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c56edb-bd9f-428c-8c17-c8c667940050 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:50.681915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c56edb-bd9f-428c-8c17-c8c667940050 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:50.681915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c56edb-bd9f-428c-8c17-c8c667940050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:50.690247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9328746-c8a3-4444-b465-e55664b3e07f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:50.701998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5f27f-11c6-4e87-bf94-c3fca8ff00ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:50.702794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5f27f-11c6-4e87-bf94-c3fca8ff00ea service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:50.703076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5f27f-11c6-4e87-bf94-c3fca8ff00ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:50.703894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b5f27f-11c6-4e87-bf94-c3fca8ff00ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:50.705898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c56edb-bd9f-428c-8c17-c8c667940050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:50.707680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c56edb-bd9f-428c-8c17-c8c667940050 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:50.708789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a8b5f27f-11c6-4e87-bf94-c3fca8ff00ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:50.736256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 913/1814] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:50.822789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5f27f-11c6-4e87-bf94-c3fca8ff00ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:50.828999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b5f27f-11c6-4e87-bf94-c3fca8ff00ea service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:50.841963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 902/1815] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:10:52.029361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c56edb-bd9f-428c-8c17-c8c667940050 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:52.031007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-69c56edb-bd9f-428c-8c17-c8c667940050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:52.070443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c56edb-bd9f-428c-8c17-c8c667940050 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:52.077230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:52.078045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:52.078550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:52.079057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] PATH_INFO: `/v3/projects/a33746b410c742d9b46e4e930a2b32ed` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:52.081199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=a33746b410c742d9b46e4e930a2b32ed)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:52.083215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:52.154179 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a33746b410c742d9b46e4e930a2b32ed', 'request_id': 'req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.163645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a33746b410c742d9b46e4e930a2b32ed', 'request_id': 'req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.172839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a33746b410c742d9b46e4e930a2b32ed', 'request_id': 'req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.190495 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] The token cache is being invalidate because project a33746b410c742d9b46e4e930a2b32ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:52.190791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.191748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.193798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 914/1816] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:52 2022] DELETE /identity/v3/projects/a33746b410c742d9b46e4e930a2b32ed => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:52.198731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5f27f-11c6-4e87-bf94-c3fca8ff00ea service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:52.210377 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b5f27f-11c6-4e87-bf94-c3fca8ff00ea service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:52.236538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:52.308571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:52.309392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:52.309741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:52.310110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] PATH_INFO: `/v3/users/6070d5ea9dcc478f8cf81d82b4aff4e5` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:52.311415 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6070d5ea9dcc478f8cf81d82b4aff4e5)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:52.313951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:52.364788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:52.366423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:52.389676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2f91bc-8b32-4353-a60e-b4bb872aa1ee admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:52.398963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:52.399683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:52.400001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:52.400443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] PATH_INFO: `/v3/projects/be02d47a320647c9be5d68df2a1d941d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:52.402221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be02d47a320647c9be5d68df2a1d941d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:52.404637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:52.416106 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6070d5ea9dcc478f8cf81d82b4aff4e5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.416840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6070d5ea9dcc478f8cf81d82b4aff4e5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.425066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6070d5ea9dcc478f8cf81d82b4aff4e5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.425831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6070d5ea9dcc478f8cf81d82b4aff4e5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.441762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6070d5ea9dcc478f8cf81d82b4aff4e5'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.456361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:52.468182 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 903/1817] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:52 2022] DELETE /identity/v3/users/6070d5ea9dcc478f8cf81d82b4aff4e5 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:52.471319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:52.471319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:52.475890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be02d47a320647c9be5d68df2a1d941d', 'request_id': 'req-808158e1-fc4b-459e-a558-dbf5935b75e4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.482829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be02d47a320647c9be5d68df2a1d941d', 'request_id': 'req-808158e1-fc4b-459e-a558-dbf5935b75e4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.489549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be02d47a320647c9be5d68df2a1d941d', 'request_id': 'req-808158e1-fc4b-459e-a558-dbf5935b75e4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.504754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] The token cache is being invalidate because project be02d47a320647c9be5d68df2a1d941d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:52.505053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.505760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.507604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 915/1818] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:52 2022] DELETE /identity/v3/projects/be02d47a320647c9be5d68df2a1d941d => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:10:52.512603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:52.515423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:52.548195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432f889b-5c5a-4981-b58c-d45d33557061 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:52.554434 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:52.555382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:52.555672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:52.556195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] PATH_INFO: `/v3/projects/ba3d716a89394d21a5c772aae6d6472d` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:52.558400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba3d716a89394d21a5c772aae6d6472d)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:52.560340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:52.594268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808158e1-fc4b-459e-a558-dbf5935b75e4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:52.599807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:52.600609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:52.600920 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:52.601245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] PATH_INFO: `/v3/users/8798ed52ba4a4ef191841c203c9d38ea` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:52.602437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8798ed52ba4a4ef191841c203c9d38ea)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:52.604922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:52.615039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:52.624039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba3d716a89394d21a5c772aae6d6472d', 'request_id': 'req-fb9e4671-92e0-43c4-827b-879e4f5421d9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.638468 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba3d716a89394d21a5c772aae6d6472d', 'request_id': 'req-fb9e4671-92e0-43c4-827b-879e4f5421d9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.647844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba3d716a89394d21a5c772aae6d6472d', 'request_id': 'req-fb9e4671-92e0-43c4-827b-879e4f5421d9'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.662126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] The token cache is being invalidate because project ba3d716a89394d21a5c772aae6d6472d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:52.662430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.663169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.667649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 904/1819] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:52 2022] DELETE /identity/v3/projects/ba3d716a89394d21a5c772aae6d6472d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:52.669330 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8798ed52ba4a4ef191841c203c9d38ea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.670017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8798ed52ba4a4ef191841c203c9d38ea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.670902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8798ed52ba4a4ef191841c203c9d38ea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.671554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:52.673811 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:52.685645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8798ed52ba4a4ef191841c203c9d38ea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.693516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8798ed52ba4a4ef191841c203c9d38ea'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.709195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 916/1820] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:52 2022] DELETE /identity/v3/users/8798ed52ba4a4ef191841c203c9d38ea => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:52.750545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e4671-92e0-43c4-827b-879e4f5421d9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:52.777946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:52.781572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:52.853990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff204ed8-8f0b-4df0-bf51-e6587ab7a8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:52.855293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff204ed8-8f0b-4df0-bf51-e6587ab7a8db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:52.855720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff204ed8-8f0b-4df0-bf51-e6587ab7a8db admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:52.856159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff204ed8-8f0b-4df0-bf51-e6587ab7a8db admin admin] PATH_INFO: `/v3/users/d4c5935b1ead4f229f1a19fcc8959fa8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:52.857736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff204ed8-8f0b-4df0-bf51-e6587ab7a8db admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c5935b1ead4f229f1a19fcc8959fa8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:52.858062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:52.860405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff204ed8-8f0b-4df0-bf51-e6587ab7a8db admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:52.894337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa56130-b571-47c8-8270-7600f6b4477e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:52.902800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:52.903865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:52.904190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:52.905669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] PATH_INFO: `/v3/projects/0df84370946e444789407b8ac0e76e9c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:52.907750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df84370946e444789407b8ac0e76e9c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:52.909923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:52.956663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ff204ed8-8f0b-4df0-bf51-e6587ab7a8db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c5935b1ead4f229f1a19fcc8959fa8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.957435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ff204ed8-8f0b-4df0-bf51-e6587ab7a8db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c5935b1ead4f229f1a19fcc8959fa8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.971181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ff204ed8-8f0b-4df0-bf51-e6587ab7a8db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c5935b1ead4f229f1a19fcc8959fa8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.972403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ff204ed8-8f0b-4df0-bf51-e6587ab7a8db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c5935b1ead4f229f1a19fcc8959fa8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.986832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ff204ed8-8f0b-4df0-bf51-e6587ab7a8db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c5935b1ead4f229f1a19fcc8959fa8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.987928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df84370946e444789407b8ac0e76e9c', 'request_id': 'req-ca0e0fad-1901-4427-875e-25be292484e8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:52.990889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:52.997525 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df84370946e444789407b8ac0e76e9c', 'request_id': 'req-ca0e0fad-1901-4427-875e-25be292484e8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.004514 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 905/1821] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:52 2022] DELETE /identity/v3/users/d4c5935b1ead4f229f1a19fcc8959fa8 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:10:53.005347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df84370946e444789407b8ac0e76e9c', 'request_id': 'req-ca0e0fad-1901-4427-875e-25be292484e8'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.018968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] The token cache is being invalidate because project 0df84370946e444789407b8ac0e76e9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:53.019261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.019938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-ca0e0fad-1901-4427-875e-25be292484e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.025752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 917/1822] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:52 2022] DELETE /identity/v3/projects/0df84370946e444789407b8ac0e76e9c => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:53.030868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2ab3a-f7f1-4287-8acc-69ce33044ffa None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.031367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2ab3a-f7f1-4287-8acc-69ce33044ffa None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.031645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2ab3a-f7f1-4287-8acc-69ce33044ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.061238 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1ba2ab3a-f7f1-4287-8acc-69ce33044ffa None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:53.062692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1ba2ab3a-f7f1-4287-8acc-69ce33044ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:53.139218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 906/1823] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:53.146200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb1711-e3ba-470e-8ddf-4e9949128287 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.146673 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb1711-e3ba-470e-8ddf-4e9949128287 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.146750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb1711-e3ba-470e-8ddf-4e9949128287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.169894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-dcfb1711-e3ba-470e-8ddf-4e9949128287 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:53.171138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dcfb1711-e3ba-470e-8ddf-4e9949128287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:53.181168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 918/1824] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:53.185929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2ab3a-f7f1-4287-8acc-69ce33044ffa None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:53.197888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2ab3a-f7f1-4287-8acc-69ce33044ffa None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:53.206574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150d660-c9ab-4099-a1f1-24a7756b0bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:53.207336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150d660-c9ab-4099-a1f1-24a7756b0bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.207739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150d660-c9ab-4099-a1f1-24a7756b0bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.208124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150d660-c9ab-4099-a1f1-24a7756b0bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.209077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150d660-c9ab-4099-a1f1-24a7756b0bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:53.209224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150d660-c9ab-4099-a1f1-24a7756b0bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:53.211293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e150d660-c9ab-4099-a1f1-24a7756b0bc5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:53.232237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 907/1825] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:53.233382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb1711-e3ba-470e-8ddf-4e9949128287 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:53.247150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150d660-c9ab-4099-a1f1-24a7756b0bc5 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:53.249258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e150d660-c9ab-4099-a1f1-24a7756b0bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:53.253017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb1711-e3ba-470e-8ddf-4e9949128287 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:53.258354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3d231-f607-4c2f-9ac8-ee74a5567165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:53.259287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3d231-f607-4c2f-9ac8-ee74a5567165 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.260169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3d231-f607-4c2f-9ac8-ee74a5567165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.260753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e3d231-f607-4c2f-9ac8-ee74a5567165 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.262203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3d231-f607-4c2f-9ac8-ee74a5567165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:53.263722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e3d231-f607-4c2f-9ac8-ee74a5567165 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:53.269624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150d660-c9ab-4099-a1f1-24a7756b0bc5 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:53.278374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:53.279141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.279547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.280247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] PATH_INFO: `/v3/projects/ed0c44c97e7d4759ab6e2af00b25b316` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.282139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed0c44c97e7d4759ab6e2af00b25b316)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:53.285681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:53.301844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 919/1826] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:53 2022] POST /identity/v3/projects => generated 409 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:10:53.306235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3d231-f607-4c2f-9ac8-ee74a5567165 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:53.360276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed0c44c97e7d4759ab6e2af00b25b316', 'request_id': 'req-d88fcb57-d2bf-4f51-b711-eed2dec35165'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.371502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed0c44c97e7d4759ab6e2af00b25b316', 'request_id': 'req-d88fcb57-d2bf-4f51-b711-eed2dec35165'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.390248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed0c44c97e7d4759ab6e2af00b25b316', 'request_id': 'req-d88fcb57-d2bf-4f51-b711-eed2dec35165'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.406328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e3d231-f607-4c2f-9ac8-ee74a5567165 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:53.418325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] The token cache is being invalidate because project ed0c44c97e7d4759ab6e2af00b25b316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:53.418631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.419497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.426797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 908/1827] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:53 2022] DELETE /identity/v3/projects/ed0c44c97e7d4759ab6e2af00b25b316 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:53.428502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:53.432952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:53.504804 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:53.529132 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cd156c-68fd-4571-b9b2-73bd0ec204bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:53.529877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd156c-68fd-4571-b9b2-73bd0ec204bc admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.530135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd156c-68fd-4571-b9b2-73bd0ec204bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.530383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cd156c-68fd-4571-b9b2-73bd0ec204bc admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.531043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd156c-68fd-4571-b9b2-73bd0ec204bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:53.532691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88fcb57-d2bf-4f51-b711-eed2dec35165 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:53.533215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cd156c-68fd-4571-b9b2-73bd0ec204bc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:53.538666 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:53.540029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.540205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.540669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] PATH_INFO: `/v3/projects/a7c5f821a7124b3994111ff6a24adf4a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.542668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7c5f821a7124b3994111ff6a24adf4a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:53.545206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:53.558974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:53.562005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 920/1828] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:53 2022] POST /identity/v3/users => generated 305 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:53.563264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85280e7-bca5-4d40-ad2e-a579b84c4a54 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.563547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85280e7-bca5-4d40-ad2e-a579b84c4a54 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.563814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85280e7-bca5-4d40-ad2e-a579b84c4a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.587126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f85280e7-bca5-4d40-ad2e-a579b84c4a54 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:53.589407 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f85280e7-bca5-4d40-ad2e-a579b84c4a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:53.599051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 921/1829] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:53.600056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85280e7-bca5-4d40-ad2e-a579b84c4a54 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:53.602549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f85280e7-bca5-4d40-ad2e-a579b84c4a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:53.617864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85280e7-bca5-4d40-ad2e-a579b84c4a54 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:53.630836 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7c5f821a7124b3994111ff6a24adf4a', 'request_id': 'req-fdae7145-4685-4636-958d-8b85dd00b4be'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.653050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7c5f821a7124b3994111ff6a24adf4a', 'request_id': 'req-fdae7145-4685-4636-958d-8b85dd00b4be'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.661404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7c5f821a7124b3994111ff6a24adf4a', 'request_id': 'req-fdae7145-4685-4636-958d-8b85dd00b4be'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.672544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] The token cache is being invalidate because project a7c5f821a7124b3994111ff6a24adf4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:53.672850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.673856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fdae7145-4685-4636-958d-8b85dd00b4be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:53.675652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 909/1830] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:53 2022] DELETE /identity/v3/projects/a7c5f821a7124b3994111ff6a24adf4a => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:53.676626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e03ba-02c3-4df7-9824-bf1802f2be62 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.676905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e03ba-02c3-4df7-9824-bf1802f2be62 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.677206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e03ba-02c3-4df7-9824-bf1802f2be62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.689011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08685d42-7bc8-4894-9bda-5223a119d752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:53.689657 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08685d42-7bc8-4894-9bda-5223a119d752 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.689928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08685d42-7bc8-4894-9bda-5223a119d752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.690201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08685d42-7bc8-4894-9bda-5223a119d752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.690841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08685d42-7bc8-4894-9bda-5223a119d752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:53.692536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08685d42-7bc8-4894-9bda-5223a119d752 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:53.701137 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-013e03ba-02c3-4df7-9824-bf1802f2be62 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:53.702369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-013e03ba-02c3-4df7-9824-bf1802f2be62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:53.707930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 922/1831] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:53.710087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c144-51e2-479c-ae25-d7da214ca800 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.710917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c144-51e2-479c-ae25-d7da214ca800 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.711384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c144-51e2-479c-ae25-d7da214ca800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.718029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 910/1832] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:53.718029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e03ba-02c3-4df7-9824-bf1802f2be62 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:53.718029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-013e03ba-02c3-4df7-9824-bf1802f2be62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:53.730598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e03ba-02c3-4df7-9824-bf1802f2be62 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:53.735977 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9889f-a54f-4a8e-84d3-2437453f120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:53.736589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9889f-a54f-4a8e-84d3-2437453f120e admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.736865 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9889f-a54f-4a8e-84d3-2437453f120e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.737246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9889f-a54f-4a8e-84d3-2437453f120e admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.737894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9889f-a54f-4a8e-84d3-2437453f120e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:53.739492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9889f-a54f-4a8e-84d3-2437453f120e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:53.739862 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cbf2c144-51e2-479c-ae25-d7da214ca800 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:53.741233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cbf2c144-51e2-479c-ae25-d7da214ca800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:53.751490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 923/1833] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:53.755130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 911/1834] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:53.755250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c144-51e2-479c-ae25-d7da214ca800 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:53.757031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9889f-a54f-4a8e-84d3-2437453f120e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:53.769243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c144-51e2-479c-ae25-d7da214ca800 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:53.771916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9889f-a54f-4a8e-84d3-2437453f120e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:53.776388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a162f-21b8-4937-a43e-905c20f5b9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:53.777212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a162f-21b8-4937-a43e-905c20f5b9ed admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.777517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01c6a6-b1ad-4d4d-b080-c75f8c35769c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:53.777885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a162f-21b8-4937-a43e-905c20f5b9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.778196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a162f-21b8-4937-a43e-905c20f5b9ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.778885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01c6a6-b1ad-4d4d-b080-c75f8c35769c admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.779098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a162f-21b8-4937-a43e-905c20f5b9ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:53.779212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01c6a6-b1ad-4d4d-b080-c75f8c35769c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.779438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01c6a6-b1ad-4d4d-b080-c75f8c35769c admin admin] PATH_INFO: `/v3/projects/078cd506dbf7497ebbe9bedde36099c4/users/9a6230338a2e458fb1867d63fb054cd8/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.779551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a162f-21b8-4937-a43e-905c20f5b9ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:53.781484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a162f-21b8-4937-a43e-905c20f5b9ed admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:53.795250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01c6a6-b1ad-4d4d-b080-c75f8c35769c admin admin] RBAC: Authorizing `identity:create_grant(project_id=078cd506dbf7497ebbe9bedde36099c4, user_id=9a6230338a2e458fb1867d63fb054cd8, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:53.797987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae01c6a6-b1ad-4d4d-b080-c75f8c35769c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:53.801225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 924/1835] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:53.801225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffed05c-fc60-4cef-a9d0-8ca6b188c65a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.801566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffed05c-fc60-4cef-a9d0-8ca6b188c65a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.801566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffed05c-fc60-4cef-a9d0-8ca6b188c65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.824602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 912/1836] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:53 2022] PUT /identity/v3/projects/078cd506dbf7497ebbe9bedde36099c4/users/9a6230338a2e458fb1867d63fb054cd8/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:53.824602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01c6a6-b1ad-4d4d-b080-c75f8c35769c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:53.825781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1ffed05c-fc60-4cef-a9d0-8ca6b188c65a None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:53.826990 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1ffed05c-fc60-4cef-a9d0-8ca6b188c65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:53.889333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01c6a6-b1ad-4d4d-b080-c75f8c35769c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:53.889920 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 925/1837] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:53.890981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055ca45-629a-4c7f-a264-5a65154d358d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.891265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055ca45-629a-4c7f-a264-5a65154d358d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.891536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055ca45-629a-4c7f-a264-5a65154d358d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.897353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b71e5b-e630-4165-b8c8-c7cd4ffe83ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:53.897955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b71e5b-e630-4165-b8c8-c7cd4ffe83ba admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:53.898238 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b71e5b-e630-4165-b8c8-c7cd4ffe83ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:53.898515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b71e5b-e630-4165-b8c8-c7cd4ffe83ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:53.899146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b71e5b-e630-4165-b8c8-c7cd4ffe83ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:53.902038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b71e5b-e630-4165-b8c8-c7cd4ffe83ba admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:53.920783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8055ca45-629a-4c7f-a264-5a65154d358d None None] MFA Rules not processed for user `9a6230338a2e458fb1867d63fb054cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:53.922090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8055ca45-629a-4c7f-a264-5a65154d358d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:53.933608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 913/1838] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:53 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:53.934382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b71e5b-e630-4165-b8c8-c7cd4ffe83ba admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:53.967072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:53.976929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:53.997458 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 926/1839] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:53 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:53.998391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055ca45-629a-4c7f-a264-5a65154d358d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.000324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b71e5b-e630-4165-b8c8-c7cd4ffe83ba admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.006339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31a177-e93c-4e47-ad05-b5f15e3a4d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.006943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a177-e93c-4e47-ad05-b5f15e3a4d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.007224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a177-e93c-4e47-ad05-b5f15e3a4d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.007494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a177-e93c-4e47-ad05-b5f15e3a4d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.008136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31a177-e93c-4e47-ad05-b5f15e3a4d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:54.008438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31a177-e93c-4e47-ad05-b5f15e3a4d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.008791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055ca45-629a-4c7f-a264-5a65154d358d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.010926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31a177-e93c-4e47-ad05-b5f15e3a4d05 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.015407 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27e66e-36ab-4cd8-81eb-20f0fef826e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.016163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27e66e-36ab-4cd8-81eb-20f0fef826e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.016552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27e66e-36ab-4cd8-81eb-20f0fef826e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.016914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d27e66e-36ab-4cd8-81eb-20f0fef826e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.017761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27e66e-36ab-4cd8-81eb-20f0fef826e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.019936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d27e66e-36ab-4cd8-81eb-20f0fef826e8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.022693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 914/1840] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:54.023463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31a177-e93c-4e47-ad05-b5f15e3a4d05 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.035452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31a177-e93c-4e47-ad05-b5f15e3a4d05 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.042060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17565c74-ef66-44a1-9c7f-33dd8969b0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.042660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565c74-ef66-44a1-9c7f-33dd8969b0a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.043205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565c74-ef66-44a1-9c7f-33dd8969b0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.043205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17565c74-ef66-44a1-9c7f-33dd8969b0a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.043839 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565c74-ef66-44a1-9c7f-33dd8969b0a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.045813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17565c74-ef66-44a1-9c7f-33dd8969b0a8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.050708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 927/1841] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:53 2022] POST /identity/v3/users => generated 288 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:54.050708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27e66e-36ab-4cd8-81eb-20f0fef826e8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.058838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d27e66e-36ab-4cd8-81eb-20f0fef826e8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.063603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206aec1a-0503-45dc-aa10-d71b8598f200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.064206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aec1a-0503-45dc-aa10-d71b8598f200 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.064484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aec1a-0503-45dc-aa10-d71b8598f200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.064753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206aec1a-0503-45dc-aa10-d71b8598f200 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.065422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206aec1a-0503-45dc-aa10-d71b8598f200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.067300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206aec1a-0503-45dc-aa10-d71b8598f200 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.083009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 915/1842] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:54 2022] POST /identity/v3/projects => generated 407 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:54.083009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17565c74-ef66-44a1-9c7f-33dd8969b0a8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.097821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 928/1843] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:54 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:54.098614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206aec1a-0503-45dc-aa10-d71b8598f200 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.144726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17565c74-ef66-44a1-9c7f-33dd8969b0a8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.162914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206aec1a-0503-45dc-aa10-d71b8598f200 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.168683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbacdb-37dd-46b6-8a06-6ba0341a28c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.169354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbacdb-37dd-46b6-8a06-6ba0341a28c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.169636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbacdb-37dd-46b6-8a06-6ba0341a28c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.169909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfbacdb-37dd-46b6-8a06-6ba0341a28c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.170556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbacdb-37dd-46b6-8a06-6ba0341a28c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.172467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfbacdb-37dd-46b6-8a06-6ba0341a28c5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.195099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 929/1844] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:54 2022] POST /identity/v3/users => generated 310 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:54.195981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbacdb-37dd-46b6-8a06-6ba0341a28c5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.204409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442d3d1-b8f8-483e-a2e8-ad82f8c6df6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.205019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442d3d1-b8f8-483e-a2e8-ad82f8c6df6b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.205326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442d3d1-b8f8-483e-a2e8-ad82f8c6df6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.205601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b442d3d1-b8f8-483e-a2e8-ad82f8c6df6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.206237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442d3d1-b8f8-483e-a2e8-ad82f8c6df6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.207827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b442d3d1-b8f8-483e-a2e8-ad82f8c6df6b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.208385 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfbacdb-37dd-46b6-8a06-6ba0341a28c5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.213033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392de5b4-37ff-40ea-844b-1d409719ca1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.213645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392de5b4-37ff-40ea-844b-1d409719ca1e admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.213927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392de5b4-37ff-40ea-844b-1d409719ca1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.214198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392de5b4-37ff-40ea-844b-1d409719ca1e admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.214834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392de5b4-37ff-40ea-844b-1d409719ca1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.216715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392de5b4-37ff-40ea-844b-1d409719ca1e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.218167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 916/1845] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:54.218825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442d3d1-b8f8-483e-a2e8-ad82f8c6df6b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.231394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b442d3d1-b8f8-483e-a2e8-ad82f8c6df6b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.236485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fb654-1a49-409a-a3cb-e9e4a2c7b348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.237079 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fb654-1a49-409a-a3cb-e9e4a2c7b348 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.237391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fb654-1a49-409a-a3cb-e9e4a2c7b348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.237664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637fb654-1a49-409a-a3cb-e9e4a2c7b348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.238296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fb654-1a49-409a-a3cb-e9e4a2c7b348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.239906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637fb654-1a49-409a-a3cb-e9e4a2c7b348 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.245022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 930/1846] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:54 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:54.245022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392de5b4-37ff-40ea-844b-1d409719ca1e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.251271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 917/1847] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:54.252117 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fb654-1a49-409a-a3cb-e9e4a2c7b348 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.257275 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392de5b4-37ff-40ea-844b-1d409719ca1e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.262173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a1dd-a9f6-4646-ac98-21176efb0881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.262798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a1dd-a9f6-4646-ac98-21176efb0881 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.263077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a1dd-a9f6-4646-ac98-21176efb0881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.263350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb8a1dd-a9f6-4646-ac98-21176efb0881 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.264008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8a1dd-a9f6-4646-ac98-21176efb0881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.265640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb8a1dd-a9f6-4646-ac98-21176efb0881 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.270853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637fb654-1a49-409a-a3cb-e9e4a2c7b348 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.275730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff45ca-6468-4836-af81-c2e1d1e60c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.276336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff45ca-6468-4836-af81-c2e1d1e60c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.276614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff45ca-6468-4836-af81-c2e1d1e60c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.276886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff45ca-6468-4836-af81-c2e1d1e60c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.277582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff45ca-6468-4836-af81-c2e1d1e60c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.279169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff45ca-6468-4836-af81-c2e1d1e60c6a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.280824 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 931/1848] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:54.282235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a1dd-a9f6-4646-ac98-21176efb0881 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.292859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 918/1849] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:54.293825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff45ca-6468-4836-af81-c2e1d1e60c6a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.294251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb8a1dd-a9f6-4646-ac98-21176efb0881 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.301335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19805bbc-5719-4bd3-bfab-37a45de923b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.301999 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19805bbc-5719-4bd3-bfab-37a45de923b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.302280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19805bbc-5719-4bd3-bfab-37a45de923b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.302551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19805bbc-5719-4bd3-bfab-37a45de923b2 admin admin] PATH_INFO: `/v3/projects/52b74f338b0d491d9d0fca49f27864cb/users/e8af76cee0d549de9f13c88ecbc7f47a/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.309057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff45ca-6468-4836-af81-c2e1d1e60c6a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.321240 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b5031-5c0b-4ec7-b0f9-05d6010e2b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.321941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b5031-5c0b-4ec7-b0f9-05d6010e2b6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.322231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b5031-5c0b-4ec7-b0f9-05d6010e2b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.322501 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b5031-5c0b-4ec7-b0f9-05d6010e2b6b admin admin] PATH_INFO: `/v3/projects/85fa4086b3b24cb385a37cc8d023fbda/users/83a9b776714a4847b4737c9bc8bbba6d/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.331242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19805bbc-5719-4bd3-bfab-37a45de923b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b74f338b0d491d9d0fca49f27864cb, user_id=e8af76cee0d549de9f13c88ecbc7f47a, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.334132 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19805bbc-5719-4bd3-bfab-37a45de923b2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.341528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b5031-5c0b-4ec7-b0f9-05d6010e2b6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=85fa4086b3b24cb385a37cc8d023fbda, user_id=83a9b776714a4847b4737c9bc8bbba6d, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.344410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b5031-5c0b-4ec7-b0f9-05d6010e2b6b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.350785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 932/1850] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:54 2022] PUT /identity/v3/projects/52b74f338b0d491d9d0fca49f27864cb/users/e8af76cee0d549de9f13c88ecbc7f47a/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:54.351612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19805bbc-5719-4bd3-bfab-37a45de923b2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.364184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 919/1851] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:54 2022] PUT /identity/v3/projects/85fa4086b3b24cb385a37cc8d023fbda/users/83a9b776714a4847b4737c9bc8bbba6d/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:54.365327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b5031-5c0b-4ec7-b0f9-05d6010e2b6b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.381813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:54.411134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:54.454013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b5031-5c0b-4ec7-b0f9-05d6010e2b6b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.454575 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19805bbc-5719-4bd3-bfab-37a45de923b2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.459505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7055595-a093-42d8-a81d-e09248a1c163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.460021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d4198-9db4-47a8-a34b-ad2a05ee5729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.460143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055595-a093-42d8-a81d-e09248a1c163 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.460370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055595-a093-42d8-a81d-e09248a1c163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.460642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7055595-a093-42d8-a81d-e09248a1c163 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.461314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7055595-a093-42d8-a81d-e09248a1c163 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.461654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d4198-9db4-47a8-a34b-ad2a05ee5729 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.462116 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d4198-9db4-47a8-a34b-ad2a05ee5729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.463266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d4198-9db4-47a8-a34b-ad2a05ee5729 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.463514 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7055595-a093-42d8-a81d-e09248a1c163 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.464434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d4198-9db4-47a8-a34b-ad2a05ee5729 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.466956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d4198-9db4-47a8-a34b-ad2a05ee5729 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.475485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 920/1852] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:54.475485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b9d24-ccc3-49e9-a587-71927b404b1d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.475485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b9d24-ccc3-49e9-a587-71927b404b1d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.475485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b9d24-ccc3-49e9-a587-71927b404b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.484498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 933/1853] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:54.484498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d4198-9db4-47a8-a34b-ad2a05ee5729 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.493900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d4198-9db4-47a8-a34b-ad2a05ee5729 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.498901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcd30a-398e-47f7-8bc4-7b67c4fcbb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.499600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcd30a-398e-47f7-8bc4-7b67c4fcbb58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.499881 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcd30a-398e-47f7-8bc4-7b67c4fcbb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.500162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbcd30a-398e-47f7-8bc4-7b67c4fcbb58 admin admin] PATH_INFO: `/v3/projects/52b74f338b0d491d9d0fca49f27864cb/users/e8af76cee0d549de9f13c88ecbc7f47a/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.502634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcd30a-398e-47f7-8bc4-7b67c4fcbb58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52b74f338b0d491d9d0fca49f27864cb, user_id=e8af76cee0d549de9f13c88ecbc7f47a, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.504816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbcd30a-398e-47f7-8bc4-7b67c4fcbb58 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.512090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-525b9d24-ccc3-49e9-a587-71927b404b1d None None] MFA Rules not processed for user `83a9b776714a4847b4737c9bc8bbba6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:54.513421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-525b9d24-ccc3-49e9-a587-71927b404b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:54.527393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 934/1854] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:54 2022] PUT /identity/v3/projects/52b74f338b0d491d9d0fca49f27864cb/users/e8af76cee0d549de9f13c88ecbc7f47a/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:54.527393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcd30a-398e-47f7-8bc4-7b67c4fcbb58 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.587280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbcd30a-398e-47f7-8bc4-7b67c4fcbb58 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.592592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011084e9-d113-431b-89e3-d9077e6e38d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.593300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011084e9-d113-431b-89e3-d9077e6e38d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.593577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011084e9-d113-431b-89e3-d9077e6e38d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.593849 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011084e9-d113-431b-89e3-d9077e6e38d0 admin admin] PATH_INFO: `/v3/projects/03892de8d1ad4e7e8395e07e56f5f0d4/users/5ed2df6d8ea04fbc85628007dddc945a/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.607763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 921/1855] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:54.608390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011084e9-d113-431b-89e3-d9077e6e38d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03892de8d1ad4e7e8395e07e56f5f0d4, user_id=5ed2df6d8ea04fbc85628007dddc945a, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.609014 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb549a60-95ed-4b7f-aca6-06fb3862af77 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.609333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb549a60-95ed-4b7f-aca6-06fb3862af77 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.609608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb549a60-95ed-4b7f-aca6-06fb3862af77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.611082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011084e9-d113-431b-89e3-d9077e6e38d0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.627517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 935/1856] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:54 2022] PUT /identity/v3/projects/03892de8d1ad4e7e8395e07e56f5f0d4/users/5ed2df6d8ea04fbc85628007dddc945a/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:54.628429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011084e9-d113-431b-89e3-d9077e6e38d0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.630057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-011084e9-d113-431b-89e3-d9077e6e38d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:54.640640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fb549a60-95ed-4b7f-aca6-06fb3862af77 None None] MFA Rules not processed for user `e8af76cee0d549de9f13c88ecbc7f47a`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:54.641882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fb549a60-95ed-4b7f-aca6-06fb3862af77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:54.698247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011084e9-d113-431b-89e3-d9077e6e38d0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.702028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c4469-952e-4bb7-8f33-31771f08364c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.702640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c4469-952e-4bb7-8f33-31771f08364c service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.702919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c4469-952e-4bb7-8f33-31771f08364c service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.703196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c4469-952e-4bb7-8f33-31771f08364c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.715477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 922/1857] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:54 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:54.723059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380233a3-dbda-4dfb-b2a4-2ca8b0c9a383 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.723643 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380233a3-dbda-4dfb-b2a4-2ca8b0c9a383 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.724126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380233a3-dbda-4dfb-b2a4-2ca8b0c9a383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.756476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-380233a3-dbda-4dfb-b2a4-2ca8b0c9a383 None None] MFA Rules not processed for user `5ed2df6d8ea04fbc85628007dddc945a`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:54.757697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-380233a3-dbda-4dfb-b2a4-2ca8b0c9a383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:54.764234 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:54.775170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:54.779057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c4469-952e-4bb7-8f33-31771f08364c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.783467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c4469-952e-4bb7-8f33-31771f08364c service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.796715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 936/1858] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:54.797770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c4469-952e-4bb7-8f33-31771f08364c service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.799596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-181c4469-952e-4bb7-8f33-31771f08364c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:54.814372 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c4469-952e-4bb7-8f33-31771f08364c service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.819886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191c97f-bd9f-4ce4-860b-c28ec5c51764 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.820575 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191c97f-bd9f-4ce4-860b-c28ec5c51764 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.820860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191c97f-bd9f-4ce4-860b-c28ec5c51764 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.821160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191c97f-bd9f-4ce4-860b-c28ec5c51764 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.832647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191c97f-bd9f-4ce4-860b-c28ec5c51764 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.834276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191c97f-bd9f-4ce4-860b-c28ec5c51764 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.843873 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 923/1859] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:54.844874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 937/1860] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:54 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:54.848257 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380233a3-dbda-4dfb-b2a4-2ca8b0c9a383 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:54.932998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380233a3-dbda-4dfb-b2a4-2ca8b0c9a383 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:54.937787 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ec35d-63cd-45e2-949f-851a232ff634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:54.938397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ec35d-63cd-45e2-949f-851a232ff634 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:54.938681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ec35d-63cd-45e2-949f-851a232ff634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:54.938960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ec35d-63cd-45e2-949f-851a232ff634 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:54.939597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ec35d-63cd-45e2-949f-851a232ff634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:54.941523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ec35d-63cd-45e2-949f-851a232ff634 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:54.976755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 924/1861] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:54 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:10:54.981731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191c97f-bd9f-4ce4-860b-c28ec5c51764 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:55.002789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191c97f-bd9f-4ce4-860b-c28ec5c51764 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:55.079837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ec35d-63cd-45e2-949f-851a232ff634 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:55.088166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0df63-0783-492e-9f2a-a89fea8af03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:55.089281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0df63-0783-492e-9f2a-a89fea8af03b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:55.089770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0df63-0783-492e-9f2a-a89fea8af03b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:55.090283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0df63-0783-492e-9f2a-a89fea8af03b admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:55.091162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0df63-0783-492e-9f2a-a89fea8af03b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:55.093394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0df63-0783-492e-9f2a-a89fea8af03b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:55.123621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 938/1862] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:54 2022] POST /identity/v3/users => generated 298 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:10:55.127501 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0df63-0783-492e-9f2a-a89fea8af03b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:55.148792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0df63-0783-492e-9f2a-a89fea8af03b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:55.155943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ec35d-63cd-45e2-949f-851a232ff634 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:55.160332 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5110b-4bf6-4b98-b9a6-0671943666d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:55.160948 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5110b-4bf6-4b98-b9a6-0671943666d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:55.161251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5110b-4bf6-4b98-b9a6-0671943666d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:55.161375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60734b2-0754-43e3-aae4-62746ead513b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:55.161530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5110b-4bf6-4b98-b9a6-0671943666d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:55.162120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60734b2-0754-43e3-aae4-62746ead513b service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:55.162227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5110b-4bf6-4b98-b9a6-0671943666d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:55.162481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60734b2-0754-43e3-aae4-62746ead513b service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:55.162818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60734b2-0754-43e3-aae4-62746ead513b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:55.163743 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5110b-4bf6-4b98-b9a6-0671943666d5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:55.175044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 939/1863] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:10:55.179554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5110b-4bf6-4b98-b9a6-0671943666d5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:55.192151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5110b-4bf6-4b98-b9a6-0671943666d5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:55.197865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8d581-5be6-45ff-9e1f-9ef4107ab6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:55.198535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8d581-5be6-45ff-9e1f-9ef4107ab6f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:55.198891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8d581-5be6-45ff-9e1f-9ef4107ab6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:55.199645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8d581-5be6-45ff-9e1f-9ef4107ab6f6 admin admin] PATH_INFO: `/v3/projects/1b7bf2bbab5e47e7b67b73fb48f3780c/users/1a06eb4cdf4544539a08e5f53644bc61/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:55.215293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:55.216518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8d581-5be6-45ff-9e1f-9ef4107ab6f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b7bf2bbab5e47e7b67b73fb48f3780c, user_id=1a06eb4cdf4544539a08e5f53644bc61, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:55.219438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8d581-5be6-45ff-9e1f-9ef4107ab6f6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:55.230084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60734b2-0754-43e3-aae4-62746ead513b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:55.231533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60734b2-0754-43e3-aae4-62746ead513b service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:55.232821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 940/1864] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:55 2022] PUT /identity/v3/projects/1b7bf2bbab5e47e7b67b73fb48f3780c/users/1a06eb4cdf4544539a08e5f53644bc61/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:10:55.236711 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8d581-5be6-45ff-9e1f-9ef4107ab6f6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:55.241741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 925/1865] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:55 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:10:55.301510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8d581-5be6-45ff-9e1f-9ef4107ab6f6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:55.308199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e980d0c5-d5ce-4023-a304-059714a31ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:55.308804 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980d0c5-d5ce-4023-a304-059714a31ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:55.309080 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980d0c5-d5ce-4023-a304-059714a31ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:55.309378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e980d0c5-d5ce-4023-a304-059714a31ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:55.310024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980d0c5-d5ce-4023-a304-059714a31ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:55.311638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e980d0c5-d5ce-4023-a304-059714a31ea1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:55.322482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 941/1866] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:10:55.326592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60734b2-0754-43e3-aae4-62746ead513b service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:55.337045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60734b2-0754-43e3-aae4-62746ead513b service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:55.342121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610a865-9aa1-4109-94a5-6e1a1c6f51e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:55.342766 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610a865-9aa1-4109-94a5-6e1a1c6f51e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:55.343066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610a865-9aa1-4109-94a5-6e1a1c6f51e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:55.343454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610a865-9aa1-4109-94a5-6e1a1c6f51e1 admin admin] PATH_INFO: `/v3/projects/1b7bf2bbab5e47e7b67b73fb48f3780c/users/1a06eb4cdf4544539a08e5f53644bc61/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:55.345411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610a865-9aa1-4109-94a5-6e1a1c6f51e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b7bf2bbab5e47e7b67b73fb48f3780c, user_id=1a06eb4cdf4544539a08e5f53644bc61, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:55.348059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610a865-9aa1-4109-94a5-6e1a1c6f51e1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:55.358325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 926/1867] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:55 2022] PUT /identity/v3/projects/1b7bf2bbab5e47e7b67b73fb48f3780c/users/1a06eb4cdf4544539a08e5f53644bc61/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:10:55.369606 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca4eda-cd6c-43dd-b706-907d7e9637c6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:55.369878 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca4eda-cd6c-43dd-b706-907d7e9637c6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:55.370156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ca4eda-cd6c-43dd-b706-907d7e9637c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:55.402020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e0ca4eda-cd6c-43dd-b706-907d7e9637c6 None None] MFA Rules not processed for user `1a06eb4cdf4544539a08e5f53644bc61`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:55.403228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e0ca4eda-cd6c-43dd-b706-907d7e9637c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:55.423752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:55.482927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 942/1868] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:55 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:10:55.499413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610a865-9aa1-4109-94a5-6e1a1c6f51e1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:55.602257 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610a865-9aa1-4109-94a5-6e1a1c6f51e1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:55.613550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b88f36-a15b-479e-85a9-4666ae1d3f24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:55.614388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b88f36-a15b-479e-85a9-4666ae1d3f24 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:55.614743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b88f36-a15b-479e-85a9-4666ae1d3f24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:55.615133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b88f36-a15b-479e-85a9-4666ae1d3f24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:55.626768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b88f36-a15b-479e-85a9-4666ae1d3f24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:55.628327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b88f36-a15b-479e-85a9-4666ae1d3f24 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:55.639838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 927/1869] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:55 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:55.733267 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca4eda-cd6c-43dd-b706-907d7e9637c6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:55.742572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ca4eda-cd6c-43dd-b706-907d7e9637c6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:55.746821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff238e82-0c77-4364-aa87-3f5e47d3cd1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:55.747481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff238e82-0c77-4364-aa87-3f5e47d3cd1c service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:55.747759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff238e82-0c77-4364-aa87-3f5e47d3cd1c service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:55.748026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff238e82-0c77-4364-aa87-3f5e47d3cd1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:55.816782 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff238e82-0c77-4364-aa87-3f5e47d3cd1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:55.818459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff238e82-0c77-4364-aa87-3f5e47d3cd1c service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:55.828210 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 943/1870] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:56.111985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b88f36-a15b-479e-85a9-4666ae1d3f24 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:56.187463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b88f36-a15b-479e-85a9-4666ae1d3f24 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:56.195756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:56.196397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:56.201158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:56.201158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] PATH_INFO: `/v3/users/5ed2df6d8ea04fbc85628007dddc945a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:56.201158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ed2df6d8ea04fbc85628007dddc945a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:56.201158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:56.257449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ed2df6d8ea04fbc85628007dddc945a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:56.258129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ed2df6d8ea04fbc85628007dddc945a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:56.269747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ed2df6d8ea04fbc85628007dddc945a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:56.270404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ed2df6d8ea04fbc85628007dddc945a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:56.282626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ed2df6d8ea04fbc85628007dddc945a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:56.286180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:56.297644 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 928/1871] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:56 2022] DELETE /identity/v3/users/5ed2df6d8ea04fbc85628007dddc945a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:56.302218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664bb2a-e03b-4d59-bcfb-cc420f31e25a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:56.302673 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664bb2a-e03b-4d59-bcfb-cc420f31e25a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:56.302784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664bb2a-e03b-4d59-bcfb-cc420f31e25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:56.321493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5664bb2a-e03b-4d59-bcfb-cc420f31e25a None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:56.323214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5664bb2a-e03b-4d59-bcfb-cc420f31e25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:56.380512 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 944/1872] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:56.390865 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:56.392751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:56.449089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248d3520-2c46-4fee-85eb-22e814b6073f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:56.452713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f5099-a375-4063-aa0a-5eee105bb85a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:56.453330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f5099-a375-4063-aa0a-5eee105bb85a service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:56.453613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f5099-a375-4063-aa0a-5eee105bb85a service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:56.453883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f5099-a375-4063-aa0a-5eee105bb85a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:56.463392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f5099-a375-4063-aa0a-5eee105bb85a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:56.464819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f5099-a375-4063-aa0a-5eee105bb85a service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:56.474255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 929/1873] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:56.757721 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664bb2a-e03b-4d59-bcfb-cc420f31e25a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:56.759977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5664bb2a-e03b-4d59-bcfb-cc420f31e25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:56.772421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664bb2a-e03b-4d59-bcfb-cc420f31e25a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:56.778698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:56.779819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:56.780049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:56.780440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] PATH_INFO: `/v3/projects/03892de8d1ad4e7e8395e07e56f5f0d4` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:56.781903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03892de8d1ad4e7e8395e07e56f5f0d4)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:56.784526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:56.832756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03892de8d1ad4e7e8395e07e56f5f0d4', 'request_id': 'req-683d60de-1e68-4df9-af7b-a08422d187b7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:56.834924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:56.840562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03892de8d1ad4e7e8395e07e56f5f0d4', 'request_id': 'req-683d60de-1e68-4df9-af7b-a08422d187b7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:56.847997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03892de8d1ad4e7e8395e07e56f5f0d4', 'request_id': 'req-683d60de-1e68-4df9-af7b-a08422d187b7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:56.865908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] The token cache is being invalidate because project 03892de8d1ad4e7e8395e07e56f5f0d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:56.866398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:56.867092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:56.869477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 945/1874] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:56 2022] DELETE /identity/v3/projects/03892de8d1ad4e7e8395e07e56f5f0d4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:56.873616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f5099-a375-4063-aa0a-5eee105bb85a service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:56.887300 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f5099-a375-4063-aa0a-5eee105bb85a service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:56.948262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28039b4a-dd50-4515-b318-bed537a8cc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:56.948951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28039b4a-dd50-4515-b318-bed537a8cc5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:56.949264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28039b4a-dd50-4515-b318-bed537a8cc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:56.949600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28039b4a-dd50-4515-b318-bed537a8cc5a admin admin] PATH_INFO: `/v3/users/1a06eb4cdf4544539a08e5f53644bc61` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:56.950973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28039b4a-dd50-4515-b318-bed537a8cc5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a06eb4cdf4544539a08e5f53644bc61)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:56.952940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28039b4a-dd50-4515-b318-bed537a8cc5a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:57.008908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-28039b4a-dd50-4515-b318-bed537a8cc5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a06eb4cdf4544539a08e5f53644bc61'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:57.010303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-28039b4a-dd50-4515-b318-bed537a8cc5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a06eb4cdf4544539a08e5f53644bc61'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:57.017682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-28039b4a-dd50-4515-b318-bed537a8cc5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a06eb4cdf4544539a08e5f53644bc61'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:57.018400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-28039b4a-dd50-4515-b318-bed537a8cc5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a06eb4cdf4544539a08e5f53644bc61'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:57.032932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-28039b4a-dd50-4515-b318-bed537a8cc5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a06eb4cdf4544539a08e5f53644bc61'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:57.046963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 930/1875] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:56 2022] DELETE /identity/v3/users/1a06eb4cdf4544539a08e5f53644bc61 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:10:57.425352 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:57.426797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:57.440087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683d60de-1e68-4df9-af7b-a08422d187b7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:57.500532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:57.501190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:57.501472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:57.501745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] PATH_INFO: `/v3/projects/1b7bf2bbab5e47e7b67b73fb48f3780c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:57.502891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b7bf2bbab5e47e7b67b73fb48f3780c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:57.504776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:57.553944 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b7bf2bbab5e47e7b67b73fb48f3780c', 'request_id': 'req-20b4a891-214d-486e-9a15-0530486b7040'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:57.562570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b7bf2bbab5e47e7b67b73fb48f3780c', 'request_id': 'req-20b4a891-214d-486e-9a15-0530486b7040'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:57.568010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b7bf2bbab5e47e7b67b73fb48f3780c', 'request_id': 'req-20b4a891-214d-486e-9a15-0530486b7040'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:57.580538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] The token cache is being invalidate because project 1b7bf2bbab5e47e7b67b73fb48f3780c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:57.580857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:57.581507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-20b4a891-214d-486e-9a15-0530486b7040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:57.583035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 946/1876] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:57 2022] DELETE /identity/v3/projects/1b7bf2bbab5e47e7b67b73fb48f3780c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:57.588574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8776d3-5c21-47cf-a749-75df0a79c806 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:57.589040 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8776d3-5c21-47cf-a749-75df0a79c806 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:57.589179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8776d3-5c21-47cf-a749-75df0a79c806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:57.608855 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ec8776d3-5c21-47cf-a749-75df0a79c806 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:57.610066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ec8776d3-5c21-47cf-a749-75df0a79c806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:57.621900 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:57.671934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 931/1877] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:57.680949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade7ac-9821-4838-a83e-c12793ff681b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:57.681287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade7ac-9821-4838-a83e-c12793ff681b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:57.681566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ade7ac-9821-4838-a83e-c12793ff681b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:57.700850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e5ade7ac-9821-4838-a83e-c12793ff681b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:57.702107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e5ade7ac-9821-4838-a83e-c12793ff681b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:57.710086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 947/1878] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:57.714797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8776d3-5c21-47cf-a749-75df0a79c806 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:57.724449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8776d3-5c21-47cf-a749-75df0a79c806 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:57.730013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1713bc7-f23c-4428-868d-a0de3a19b2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:57.730618 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1713bc7-f23c-4428-868d-a0de3a19b2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:57.730893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1713bc7-f23c-4428-868d-a0de3a19b2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:57.731162 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1713bc7-f23c-4428-868d-a0de3a19b2d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:57.731804 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1713bc7-f23c-4428-868d-a0de3a19b2d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:57.732103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1713bc7-f23c-4428-868d-a0de3a19b2d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:57.733738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1713bc7-f23c-4428-868d-a0de3a19b2d4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:57.745874 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 932/1879] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:57.750316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade7ac-9821-4838-a83e-c12793ff681b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:57.759987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ade7ac-9821-4838-a83e-c12793ff681b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:57.764537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab9b01-92a0-4bf1-afb7-42082b897cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:57.765155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab9b01-92a0-4bf1-afb7-42082b897cb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:57.765437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab9b01-92a0-4bf1-afb7-42082b897cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:57.765704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ab9b01-92a0-4bf1-afb7-42082b897cb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:57.766366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab9b01-92a0-4bf1-afb7-42082b897cb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:57.768239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ab9b01-92a0-4bf1-afb7-42082b897cb8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:57.800149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 948/1880] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:57 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:57.801341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1713bc7-f23c-4428-868d-a0de3a19b2d4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:57.811687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1713bc7-f23c-4428-868d-a0de3a19b2d4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:57.866474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2657b7d-051d-4600-9eed-9bae702b42d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:57.867262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2657b7d-051d-4600-9eed-9bae702b42d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:57.867564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2657b7d-051d-4600-9eed-9bae702b42d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:57.867831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2657b7d-051d-4600-9eed-9bae702b42d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:57.868491 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2657b7d-051d-4600-9eed-9bae702b42d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:57.870425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2657b7d-051d-4600-9eed-9bae702b42d5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:57.891379 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 933/1881] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:57 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:57.895563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab9b01-92a0-4bf1-afb7-42082b897cb8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:57.906327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ab9b01-92a0-4bf1-afb7-42082b897cb8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:57.910816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1ede8-e91d-4af0-9116-2a27642e44a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:57.911402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ede8-e91d-4af0-9116-2a27642e44a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:57.911676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ede8-e91d-4af0-9116-2a27642e44a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:57.911944 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd1ede8-e91d-4af0-9116-2a27642e44a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:57.912577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1ede8-e91d-4af0-9116-2a27642e44a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:57.914222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd1ede8-e91d-4af0-9116-2a27642e44a3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:57.923855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 949/1882] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:57.927636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2657b7d-051d-4600-9eed-9bae702b42d5 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:57.937673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2657b7d-051d-4600-9eed-9bae702b42d5 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:57.942153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2435a60-2f48-4376-9433-c62dcd420f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:57.942733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2435a60-2f48-4376-9433-c62dcd420f4d admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:57.943008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2435a60-2f48-4376-9433-c62dcd420f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:57.943277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2435a60-2f48-4376-9433-c62dcd420f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:57.943903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2435a60-2f48-4376-9433-c62dcd420f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:57.945516 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2435a60-2f48-4376-9433-c62dcd420f4d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:57.956491 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 934/1883] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:57.960581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1ede8-e91d-4af0-9116-2a27642e44a3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:57.971193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd1ede8-e91d-4af0-9116-2a27642e44a3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:57.975844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73269c62-3e4d-4ba8-8edd-7a1ed8ee11db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:57.976483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73269c62-3e4d-4ba8-8edd-7a1ed8ee11db admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:57.976762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73269c62-3e4d-4ba8-8edd-7a1ed8ee11db admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:57.977035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73269c62-3e4d-4ba8-8edd-7a1ed8ee11db admin admin] PATH_INFO: `/v3/projects/25b138c4d1b84ea699b4e2cbb742652b/users/1e547dd08fd44de8abf08471d3a55eb2/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:57.991592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73269c62-3e4d-4ba8-8edd-7a1ed8ee11db admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b138c4d1b84ea699b4e2cbb742652b, user_id=1e547dd08fd44de8abf08471d3a55eb2, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:57.994266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73269c62-3e4d-4ba8-8edd-7a1ed8ee11db admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:58.007752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 950/1884] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:57 2022] PUT /identity/v3/projects/25b138c4d1b84ea699b4e2cbb742652b/users/1e547dd08fd44de8abf08471d3a55eb2/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:58.018746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c1a01-d173-4913-b964-78017461573e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:58.019038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c1a01-d173-4913-b964-78017461573e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:58.023634 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6c1a01-d173-4913-b964-78017461573e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:58.053690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6d6c1a01-d173-4913-b964-78017461573e None None] MFA Rules not processed for user `1e547dd08fd44de8abf08471d3a55eb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:58.054995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6d6c1a01-d173-4913-b964-78017461573e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:58.120042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 935/1885] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:58.131114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73269c62-3e4d-4ba8-8edd-7a1ed8ee11db admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:58.132753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-73269c62-3e4d-4ba8-8edd-7a1ed8ee11db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:58.143686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:58.191713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73269c62-3e4d-4ba8-8edd-7a1ed8ee11db admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:58.195212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ecb9c-c146-436a-970a-cb1bbe889372 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:58.195806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ecb9c-c146-436a-970a-cb1bbe889372 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:58.196078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ecb9c-c146-436a-970a-cb1bbe889372 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:58.196343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ecb9c-c146-436a-970a-cb1bbe889372 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:58.205027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ecb9c-c146-436a-970a-cb1bbe889372 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:58.206455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5ecb9c-c146-436a-970a-cb1bbe889372 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:58.215190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 951/1886] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:58.236935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c1a01-d173-4913-b964-78017461573e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:58.250942 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:58.331062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6c1a01-d173-4913-b964-78017461573e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:58.336914 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:58.337690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:58.338123 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:58.338398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] PATH_INFO: `/v3/users/1e547dd08fd44de8abf08471d3a55eb2` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:58.339626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e547dd08fd44de8abf08471d3a55eb2)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:58.341881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:58.394987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e547dd08fd44de8abf08471d3a55eb2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:58.395697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e547dd08fd44de8abf08471d3a55eb2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:58.403646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e547dd08fd44de8abf08471d3a55eb2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:58.404334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e547dd08fd44de8abf08471d3a55eb2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:58.417696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e547dd08fd44de8abf08471d3a55eb2'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:58.432619 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 936/1887] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:10:58 2022] DELETE /identity/v3/users/1e547dd08fd44de8abf08471d3a55eb2 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:58.436931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ca12-95d2-48c5-a84e-23854532c6cb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:58.437438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ca12-95d2-48c5-a84e-23854532c6cb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:58.437544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb3ca12-95d2-48c5-a84e-23854532c6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:58.460301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1cb3ca12-95d2-48c5-a84e-23854532c6cb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:58.461912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1cb3ca12-95d2-48c5-a84e-23854532c6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:58.541969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 952/1888] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:58.551511 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:58.553012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:58.612982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8397b3-5f3a-4473-8d23-611d57cd3460 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:58.616779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdbfb7-9ee0-4b82-80f0-934be8651d4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:58.617411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdbfb7-9ee0-4b82-80f0-934be8651d4d service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:58.617689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdbfb7-9ee0-4b82-80f0-934be8651d4d service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:58.617959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdbfb7-9ee0-4b82-80f0-934be8651d4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:58.629223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdbfb7-9ee0-4b82-80f0-934be8651d4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:58.629529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdbfb7-9ee0-4b82-80f0-934be8651d4d service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:58.639328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 937/1889] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:10:58.914708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3ca12-95d2-48c5-a84e-23854532c6cb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:58.916971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1cb3ca12-95d2-48c5-a84e-23854532c6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:58.929056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb3ca12-95d2-48c5-a84e-23854532c6cb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:58.934047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:58.934679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:58.934950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:58.935661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] PATH_INFO: `/v3/projects/25b138c4d1b84ea699b4e2cbb742652b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:58.936664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25b138c4d1b84ea699b4e2cbb742652b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:58.938565 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:58.984169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25b138c4d1b84ea699b4e2cbb742652b', 'request_id': 'req-b9e60ce5-7114-4ac6-bc7e-851fa526f049'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:58.990048 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25b138c4d1b84ea699b4e2cbb742652b', 'request_id': 'req-b9e60ce5-7114-4ac6-bc7e-851fa526f049'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:58.995540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25b138c4d1b84ea699b4e2cbb742652b', 'request_id': 'req-b9e60ce5-7114-4ac6-bc7e-851fa526f049'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:59.010606 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] The token cache is being invalidate because project 25b138c4d1b84ea699b4e2cbb742652b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:10:59.010894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:59.011510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b9e60ce5-7114-4ac6-bc7e-851fa526f049 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:10:59.013013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 953/1890] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:10:58 2022] DELETE /identity/v3/projects/25b138c4d1b84ea699b4e2cbb742652b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:59.018871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e5bf9-9f2e-4674-9412-6469ce907d3f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:59.019157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e5bf9-9f2e-4674-9412-6469ce907d3f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:59.019431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e5bf9-9f2e-4674-9412-6469ce907d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:59.039520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9f4e5bf9-9f2e-4674-9412-6469ce907d3f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:59.041637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9f4e5bf9-9f2e-4674-9412-6469ce907d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:59.078470 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:59.110431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 938/1891] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:59.118191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb9da-69a5-4ebe-ac0d-6c8394bc26ed None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:59.118508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb9da-69a5-4ebe-ac0d-6c8394bc26ed None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:59.118781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55deb9da-69a5-4ebe-ac0d-6c8394bc26ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:59.138192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-55deb9da-69a5-4ebe-ac0d-6c8394bc26ed None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:59.139436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-55deb9da-69a5-4ebe-ac0d-6c8394bc26ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:59.147245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 954/1892] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:59.152072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e5bf9-9f2e-4674-9412-6469ce907d3f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:59.161521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e5bf9-9f2e-4674-9412-6469ce907d3f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:59.166803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1318689-3429-46da-ad38-079e0dc6472c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:59.167385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318689-3429-46da-ad38-079e0dc6472c admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:59.167658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318689-3429-46da-ad38-079e0dc6472c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:59.167926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1318689-3429-46da-ad38-079e0dc6472c admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:59.168557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1318689-3429-46da-ad38-079e0dc6472c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:10:59.168857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1318689-3429-46da-ad38-079e0dc6472c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:59.170467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1318689-3429-46da-ad38-079e0dc6472c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:59.181983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 939/1893] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:10:59.186078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deb9da-69a5-4ebe-ac0d-6c8394bc26ed None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:59.195089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55deb9da-69a5-4ebe-ac0d-6c8394bc26ed None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:59.199678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950e9ca-dc27-4208-8e3c-7f898229a9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:59.200265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950e9ca-dc27-4208-8e3c-7f898229a9ed admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:59.200543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950e9ca-dc27-4208-8e3c-7f898229a9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:59.200813 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d950e9ca-dc27-4208-8e3c-7f898229a9ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:59.201476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950e9ca-dc27-4208-8e3c-7f898229a9ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:59.203386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d950e9ca-dc27-4208-8e3c-7f898229a9ed admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:59.230660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 955/1894] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:10:59 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:59.234811 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1318689-3429-46da-ad38-079e0dc6472c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:59.244663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1318689-3429-46da-ad38-079e0dc6472c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:59.304567 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b5918-aff4-4eab-b0d8-93f4e81a2ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:59.305209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b5918-aff4-4eab-b0d8-93f4e81a2ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:59.305489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b5918-aff4-4eab-b0d8-93f4e81a2ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:59.306173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b5918-aff4-4eab-b0d8-93f4e81a2ed1 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:59.306867 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b5918-aff4-4eab-b0d8-93f4e81a2ed1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:59.308762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b5918-aff4-4eab-b0d8-93f4e81a2ed1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:59.329465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 940/1895] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:10:59 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:10:59.333515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950e9ca-dc27-4208-8e3c-7f898229a9ed admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:59.340886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:10:59.344797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d950e9ca-dc27-4208-8e3c-7f898229a9ed admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:59.351842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d6571-be12-4fa3-a1f1-dde4bf8c428e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:59.352550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d6571-be12-4fa3-a1f1-dde4bf8c428e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:59.353225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d6571-be12-4fa3-a1f1-dde4bf8c428e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:59.353716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51d6571-be12-4fa3-a1f1-dde4bf8c428e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:59.354399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d6571-be12-4fa3-a1f1-dde4bf8c428e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:59.355991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51d6571-be12-4fa3-a1f1-dde4bf8c428e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:59.369846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 956/1896] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:59.371057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b5918-aff4-4eab-b0d8-93f4e81a2ed1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:59.381047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b5918-aff4-4eab-b0d8-93f4e81a2ed1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:59.386320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a849d909-9357-49d8-b5c8-2a2d3a3f1c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:59.386917 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849d909-9357-49d8-b5c8-2a2d3a3f1c0b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:59.387199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849d909-9357-49d8-b5c8-2a2d3a3f1c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:59.387472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a849d909-9357-49d8-b5c8-2a2d3a3f1c0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:59.388109 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849d909-9357-49d8-b5c8-2a2d3a3f1c0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:59.389753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a849d909-9357-49d8-b5c8-2a2d3a3f1c0b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:59.401492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 941/1897] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:10:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:10:59.405385 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d6571-be12-4fa3-a1f1-dde4bf8c428e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:59.415123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51d6571-be12-4fa3-a1f1-dde4bf8c428e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:59.419737 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a423e37-874f-454f-897f-fc5b3987210d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:59.420371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a423e37-874f-454f-897f-fc5b3987210d admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:59.420642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a423e37-874f-454f-897f-fc5b3987210d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:59.420912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a423e37-874f-454f-897f-fc5b3987210d admin admin] PATH_INFO: `/v3/projects/d58361c0e40c48f4966aee3515859326/users/cd913a3c0cff4787a90603f3afb2e883/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:59.435689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a423e37-874f-454f-897f-fc5b3987210d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58361c0e40c48f4966aee3515859326, user_id=cd913a3c0cff4787a90603f3afb2e883, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:59.438342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a423e37-874f-454f-897f-fc5b3987210d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:59.451486 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 957/1898] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:10:59 2022] PUT /identity/v3/projects/d58361c0e40c48f4966aee3515859326/users/cd913a3c0cff4787a90603f3afb2e883/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:10:59.456637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad7d6-25e8-4235-b9e6-8616d15b66d3 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:59.456927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad7d6-25e8-4235-b9e6-8616d15b66d3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:59.457243 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ad7d6-25e8-4235-b9e6-8616d15b66d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:59.476751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2d6ad7d6-25e8-4235-b9e6-8616d15b66d3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:10:59.478085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2d6ad7d6-25e8-4235-b9e6-8616d15b66d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:59.541305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 942/1899] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:10:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:10:59.553340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a423e37-874f-454f-897f-fc5b3987210d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:10:59.555227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3a423e37-874f-454f-897f-fc5b3987210d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:10:59.619368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a423e37-874f-454f-897f-fc5b3987210d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:10:59.624067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc98e1a-1080-464b-99b1-43688b45befd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:10:59.624677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98e1a-1080-464b-99b1-43688b45befd service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:10:59.624956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98e1a-1080-464b-99b1-43688b45befd service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:10:59.625262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc98e1a-1080-464b-99b1-43688b45befd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:10:59.635568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc98e1a-1080-464b-99b1-43688b45befd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:10:59.637013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc98e1a-1080-464b-99b1-43688b45befd service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:10:59.646729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 958/1900] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:10:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:00.335121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31b7ce-2048-44da-ab65-a95add759917 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:00.335637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31b7ce-2048-44da-ab65-a95add759917 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:00.335723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31b7ce-2048-44da-ab65-a95add759917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:00.363672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-de31b7ce-2048-44da-ab65-a95add759917 None None] MFA Rules not processed for user `cd913a3c0cff4787a90603f3afb2e883`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:00.364951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-de31b7ce-2048-44da-ab65-a95add759917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:00.395198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:00.436965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 943/1901] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:00.447923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc98e1a-1080-464b-99b1-43688b45befd service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:00.449551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9cc98e1a-1080-464b-99b1-43688b45befd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:00.510026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc98e1a-1080-464b-99b1-43688b45befd service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:00.513652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da99354-4e0e-4b01-babc-d8db4bf3677b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:00.514261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99354-4e0e-4b01-babc-d8db4bf3677b service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:00.514536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99354-4e0e-4b01-babc-d8db4bf3677b service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:00.514811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da99354-4e0e-4b01-babc-d8db4bf3677b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:00.523282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da99354-4e0e-4b01-babc-d8db4bf3677b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:00.525218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da99354-4e0e-4b01-babc-d8db4bf3677b service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:00.533945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 959/1902] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:11:01.563004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18406f8a-2ad7-4b41-9eb3-4f957473ed97 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:01.563578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18406f8a-2ad7-4b41-9eb3-4f957473ed97 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:01.563708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18406f8a-2ad7-4b41-9eb3-4f957473ed97 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:01.565152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 944/1903] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:11:01.575009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894616f-ad46-4d82-8c62-a55f3806874b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:01.575467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894616f-ad46-4d82-8c62-a55f3806874b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:01.575573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2894616f-ad46-4d82-8c62-a55f3806874b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:01.596947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2894616f-ad46-4d82-8c62-a55f3806874b None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:01.598113 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2894616f-ad46-4d82-8c62-a55f3806874b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:01.667221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 960/1904] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:11:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:01.679367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18406f8a-2ad7-4b41-9eb3-4f957473ed97 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:01.682046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-18406f8a-2ad7-4b41-9eb3-4f957473ed97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:01.699261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18406f8a-2ad7-4b41-9eb3-4f957473ed97 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:01.704738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b7292-82fa-4a18-9b79-fc451d27d7f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:01.705364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b7292-82fa-4a18-9b79-fc451d27d7f2 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:01.705642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b7292-82fa-4a18-9b79-fc451d27d7f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:01.705911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205b7292-82fa-4a18-9b79-fc451d27d7f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:01.714582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b7292-82fa-4a18-9b79-fc451d27d7f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:01.716152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205b7292-82fa-4a18-9b79-fc451d27d7f2 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:01.726235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 945/1905] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:11.656666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:11.722020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=19 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:17.805447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f86bb9-f86e-4d29-be70-cf17fe8225f4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:17.806001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f86bb9-f86e-4d29-be70-cf17fe8225f4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:17.806001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f86bb9-f86e-4d29-be70-cf17fe8225f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:17.827473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e4f86bb9-f86e-4d29-be70-cf17fe8225f4 None None] MFA Rules not processed for user `cd913a3c0cff4787a90603f3afb2e883`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:17.828697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e4f86bb9-f86e-4d29-be70-cf17fe8225f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:17.836540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 961/1906] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:19.288699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b7292-82fa-4a18-9b79-fc451d27d7f2 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:19.290632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-205b7292-82fa-4a18-9b79-fc451d27d7f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:19.304199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205b7292-82fa-4a18-9b79-fc451d27d7f2 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:19.309946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:19.310939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:19.311311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:19.311697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] PATH_INFO: `/v3/users/83a9b776714a4847b4737c9bc8bbba6d` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:19.312918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83a9b776714a4847b4737c9bc8bbba6d)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:19.315016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:19.362641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83a9b776714a4847b4737c9bc8bbba6d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:19.363291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83a9b776714a4847b4737c9bc8bbba6d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:19.370770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83a9b776714a4847b4737c9bc8bbba6d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:19.371394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83a9b776714a4847b4737c9bc8bbba6d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:19.384487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83a9b776714a4847b4737c9bc8bbba6d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:19.400498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 946/1907] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:11:19 2022] DELETE /identity/v3/users/83a9b776714a4847b4737c9bc8bbba6d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:19.405131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcde81-d122-43bc-99af-19c5c361918f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:19.405586 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcde81-d122-43bc-99af-19c5c361918f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:19.405695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fcde81-d122-43bc-99af-19c5c361918f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:19.425362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-74fcde81-d122-43bc-99af-19c5c361918f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:19.426587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-74fcde81-d122-43bc-99af-19c5c361918f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:19.487617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 962/1908] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:11:19.498075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:19.499481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:19.558212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5d481-4c9c-4668-bcbc-45d85d483174 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:19.561943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c4b61-ab28-4b62-ac43-f5d70710ecab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:19.562559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c4b61-ab28-4b62-ac43-f5d70710ecab service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:19.562840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c4b61-ab28-4b62-ac43-f5d70710ecab service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:19.563149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c4b61-ab28-4b62-ac43-f5d70710ecab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:19.572816 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c4b61-ab28-4b62-ac43-f5d70710ecab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:19.574295 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c4b61-ab28-4b62-ac43-f5d70710ecab service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:19.583928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 947/1909] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:19.879909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcde81-d122-43bc-99af-19c5c361918f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:19.881396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-74fcde81-d122-43bc-99af-19c5c361918f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:19.894884 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fcde81-d122-43bc-99af-19c5c361918f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:19.901342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:19.901832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:19.902178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:19.902580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] PATH_INFO: `/v3/projects/85fa4086b3b24cb385a37cc8d023fbda` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:19.903852 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85fa4086b3b24cb385a37cc8d023fbda)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:19.905879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:19.954494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85fa4086b3b24cb385a37cc8d023fbda', 'request_id': 'req-fe181df9-66fa-4bc1-927f-a63f2f312772'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:19.961399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85fa4086b3b24cb385a37cc8d023fbda', 'request_id': 'req-fe181df9-66fa-4bc1-927f-a63f2f312772'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:19.984896 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85fa4086b3b24cb385a37cc8d023fbda', 'request_id': 'req-fe181df9-66fa-4bc1-927f-a63f2f312772'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:19.999590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] The token cache is being invalidate because project 85fa4086b3b24cb385a37cc8d023fbda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:11:19.999879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:20.000506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fe181df9-66fa-4bc1-927f-a63f2f312772 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:20.002177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 963/1910] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:11:19 2022] DELETE /identity/v3/projects/85fa4086b3b24cb385a37cc8d023fbda => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:20.007616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a7f52-8361-414f-904a-5b3db8d40316 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:20.008027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a7f52-8361-414f-904a-5b3db8d40316 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:20.008381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a7f52-8361-414f-904a-5b3db8d40316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:20.035718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-835a7f52-8361-414f-904a-5b3db8d40316 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:20.036934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-835a7f52-8361-414f-904a-5b3db8d40316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:20.101067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 948/1911] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:20.109214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac157b9-5393-49d0-a1fa-ff88fac6ae1c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:20.109506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac157b9-5393-49d0-a1fa-ff88fac6ae1c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:20.109789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac157b9-5393-49d0-a1fa-ff88fac6ae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:20.129170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-aac157b9-5393-49d0-a1fa-ff88fac6ae1c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:20.130370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-aac157b9-5393-49d0-a1fa-ff88fac6ae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:20.138337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 964/1912] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:20.142481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a7f52-8361-414f-904a-5b3db8d40316 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:20.152590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a7f52-8361-414f-904a-5b3db8d40316 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:20.157131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69207347-daf5-4045-baeb-02401bed567f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:20.157734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69207347-daf5-4045-baeb-02401bed567f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:20.158008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69207347-daf5-4045-baeb-02401bed567f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:20.158278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69207347-daf5-4045-baeb-02401bed567f admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:20.158910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69207347-daf5-4045-baeb-02401bed567f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:11:20.159209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69207347-daf5-4045-baeb-02401bed567f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:20.160800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69207347-daf5-4045-baeb-02401bed567f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:20.172809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 949/1913] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:11:20.176987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac157b9-5393-49d0-a1fa-ff88fac6ae1c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:20.186593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac157b9-5393-49d0-a1fa-ff88fac6ae1c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:20.191276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa1ac7-19bf-4598-a605-7b4db6bcdfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:20.191871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa1ac7-19bf-4598-a605-7b4db6bcdfbc admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:20.192150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa1ac7-19bf-4598-a605-7b4db6bcdfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:20.192422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa1ac7-19bf-4598-a605-7b4db6bcdfbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:20.193083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa1ac7-19bf-4598-a605-7b4db6bcdfbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:20.195045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fa1ac7-19bf-4598-a605-7b4db6bcdfbc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:20.222477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 965/1914] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:11:20 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:20.227293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69207347-daf5-4045-baeb-02401bed567f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:20.232326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:20.238794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69207347-daf5-4045-baeb-02401bed567f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:20.302141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba94dcd-185c-4912-9df6-f28d1544be8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:20.302743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba94dcd-185c-4912-9df6-f28d1544be8c admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:20.303001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba94dcd-185c-4912-9df6-f28d1544be8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:20.303268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba94dcd-185c-4912-9df6-f28d1544be8c admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:20.303902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba94dcd-185c-4912-9df6-f28d1544be8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:20.306318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba94dcd-185c-4912-9df6-f28d1544be8c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:20.327823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 950/1915] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:11:20 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:20.332155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa1ac7-19bf-4598-a605-7b4db6bcdfbc admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:20.343047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fa1ac7-19bf-4598-a605-7b4db6bcdfbc admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:20.347628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c6132-145a-4733-8977-c61b22f22b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:20.348226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c6132-145a-4733-8977-c61b22f22b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:20.348506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c6132-145a-4733-8977-c61b22f22b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:20.348778 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40c6132-145a-4733-8977-c61b22f22b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:20.349462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c6132-145a-4733-8977-c61b22f22b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:20.351048 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40c6132-145a-4733-8977-c61b22f22b0d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:20.361307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 966/1916] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:11:20.365416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba94dcd-185c-4912-9df6-f28d1544be8c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:20.375684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba94dcd-185c-4912-9df6-f28d1544be8c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:20.380197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a446b9aa-183d-412b-bf41-685cad7063bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:20.380785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446b9aa-183d-412b-bf41-685cad7063bf admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:20.381056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446b9aa-183d-412b-bf41-685cad7063bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:20.381358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a446b9aa-183d-412b-bf41-685cad7063bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:20.381986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446b9aa-183d-412b-bf41-685cad7063bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:20.384170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a446b9aa-183d-412b-bf41-685cad7063bf admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:20.394666 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 951/1917] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:20.399791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c6132-145a-4733-8977-c61b22f22b0d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:20.409713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40c6132-145a-4733-8977-c61b22f22b0d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:20.415643 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf63a8-338c-4b6b-8c3b-f1a85efe01c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:20.416319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf63a8-338c-4b6b-8c3b-f1a85efe01c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:20.416598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf63a8-338c-4b6b-8c3b-f1a85efe01c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:20.416869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bf63a8-338c-4b6b-8c3b-f1a85efe01c6 admin admin] PATH_INFO: `/v3/projects/ee5aa87e5ba34b3cae451859b7c345c2/users/d523e68b249c44fa88cf48ae1a0ae7d8/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:20.432451 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf63a8-338c-4b6b-8c3b-f1a85efe01c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee5aa87e5ba34b3cae451859b7c345c2, user_id=d523e68b249c44fa88cf48ae1a0ae7d8, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:20.435216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bf63a8-338c-4b6b-8c3b-f1a85efe01c6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:20.458360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 967/1918] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:20 2022] PUT /identity/v3/projects/ee5aa87e5ba34b3cae451859b7c345c2/users/d523e68b249c44fa88cf48ae1a0ae7d8/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:20.470356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7b1a-d0d5-4ad2-9486-19285cb755f5 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:20.470637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7b1a-d0d5-4ad2-9486-19285cb755f5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:20.470904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafa7b1a-d0d5-4ad2-9486-19285cb755f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:20.498731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fafa7b1a-d0d5-4ad2-9486-19285cb755f5 None None] MFA Rules not processed for user `d523e68b249c44fa88cf48ae1a0ae7d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:20.499931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fafa7b1a-d0d5-4ad2-9486-19285cb755f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:20.562671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 952/1919] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:20 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:20.574481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf63a8-338c-4b6b-8c3b-f1a85efe01c6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:20.576221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-19bf63a8-338c-4b6b-8c3b-f1a85efe01c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:20.591494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:20.635402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bf63a8-338c-4b6b-8c3b-f1a85efe01c6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:20.639025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fde505-ad29-4f69-9ef0-4e91ff9ae967 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:20.639625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fde505-ad29-4f69-9ef0-4e91ff9ae967 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:20.639903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fde505-ad29-4f69-9ef0-4e91ff9ae967 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:20.640175 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fde505-ad29-4f69-9ef0-4e91ff9ae967 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:20.648297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fde505-ad29-4f69-9ef0-4e91ff9ae967 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:20.649983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fde505-ad29-4f69-9ef0-4e91ff9ae967 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:20.658598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 968/1920] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:25.746104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7b1a-d0d5-4ad2-9486-19285cb755f5 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:25.747842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fafa7b1a-d0d5-4ad2-9486-19285cb755f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:25.844543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafa7b1a-d0d5-4ad2-9486-19285cb755f5 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:25.849526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:25.850372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:25.850640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:25.850917 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] PATH_INFO: `/v3/users/9a6230338a2e458fb1867d63fb054cd8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:25.852112 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6230338a2e458fb1867d63fb054cd8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:25.855078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:25.868884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:25.908642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6230338a2e458fb1867d63fb054cd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:25.909389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6230338a2e458fb1867d63fb054cd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:25.917555 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6230338a2e458fb1867d63fb054cd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:25.918708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6230338a2e458fb1867d63fb054cd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:25.931422 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6230338a2e458fb1867d63fb054cd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:25.950855 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 953/1921] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:11:25 2022] DELETE /identity/v3/users/9a6230338a2e458fb1867d63fb054cd8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:11:25.955446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986033d7-4c2d-4184-9171-84591331a79b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:25.955966 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986033d7-4c2d-4184-9171-84591331a79b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:25.956060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986033d7-4c2d-4184-9171-84591331a79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:25.975654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-986033d7-4c2d-4184-9171-84591331a79b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:25.977006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-986033d7-4c2d-4184-9171-84591331a79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:26.045200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 969/1922] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:26.055976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:26.057575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:26.120690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61109f88-3b7f-4379-97b6-ee1e2bc48242 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:26.124369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef54cc-b8ac-4eef-ba3e-15a70657c24e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:26.124961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef54cc-b8ac-4eef-ba3e-15a70657c24e service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:26.125268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef54cc-b8ac-4eef-ba3e-15a70657c24e service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:26.125540 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef54cc-b8ac-4eef-ba3e-15a70657c24e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:26.134595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef54cc-b8ac-4eef-ba3e-15a70657c24e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:26.136001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ef54cc-b8ac-4eef-ba3e-15a70657c24e service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:26.145507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 954/1923] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:26.625683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986033d7-4c2d-4184-9171-84591331a79b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:26.627629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-986033d7-4c2d-4184-9171-84591331a79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:26.642952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986033d7-4c2d-4184-9171-84591331a79b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:26.647646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:26.648286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:26.648560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:26.648830 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] PATH_INFO: `/v3/projects/078cd506dbf7497ebbe9bedde36099c4` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:26.650807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=078cd506dbf7497ebbe9bedde36099c4)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:26.652759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:26.702135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '078cd506dbf7497ebbe9bedde36099c4', 'request_id': 'req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:26.709251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '078cd506dbf7497ebbe9bedde36099c4', 'request_id': 'req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:26.715454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '078cd506dbf7497ebbe9bedde36099c4', 'request_id': 'req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:26.729525 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] The token cache is being invalidate because project 078cd506dbf7497ebbe9bedde36099c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:11:26.729818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:26.730614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:26.732114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 970/1924] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:11:26 2022] DELETE /identity/v3/projects/078cd506dbf7497ebbe9bedde36099c4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:11:26.736480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef54cc-b8ac-4eef-ba3e-15a70657c24e service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:26.746844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ef54cc-b8ac-4eef-ba3e-15a70657c24e service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:26.802688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea3aac0-74b0-4c81-98a5-949127c06194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:26.803311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3aac0-74b0-4c81-98a5-949127c06194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:26.803585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3aac0-74b0-4c81-98a5-949127c06194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:26.803905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea3aac0-74b0-4c81-98a5-949127c06194 admin admin] PATH_INFO: `/v3/users/e8af76cee0d549de9f13c88ecbc7f47a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:26.805254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3aac0-74b0-4c81-98a5-949127c06194 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8af76cee0d549de9f13c88ecbc7f47a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:26.807831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea3aac0-74b0-4c81-98a5-949127c06194 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:26.843759 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:26.853986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0ea3aac0-74b0-4c81-98a5-949127c06194 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8af76cee0d549de9f13c88ecbc7f47a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:26.854634 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0ea3aac0-74b0-4c81-98a5-949127c06194 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8af76cee0d549de9f13c88ecbc7f47a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:26.861940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0ea3aac0-74b0-4c81-98a5-949127c06194 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8af76cee0d549de9f13c88ecbc7f47a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:26.862701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0ea3aac0-74b0-4c81-98a5-949127c06194 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8af76cee0d549de9f13c88ecbc7f47a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:26.875700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0ea3aac0-74b0-4c81-98a5-949127c06194 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8af76cee0d549de9f13c88ecbc7f47a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:26.888278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 955/1925] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:11:26 2022] DELETE /identity/v3/users/e8af76cee0d549de9f13c88ecbc7f47a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:27.156755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:27.158285 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:27.173719 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8364d0b-8b5b-4678-919d-2f8ab310f2f5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:27.213730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:27.233585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:27.234222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:27.234500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:27.234775 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] PATH_INFO: `/v3/projects/52b74f338b0d491d9d0fca49f27864cb` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:27.235897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52b74f338b0d491d9d0fca49f27864cb)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:27.237812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:27.283455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52b74f338b0d491d9d0fca49f27864cb', 'request_id': 'req-9a210daa-7864-4aac-97cc-fa1a5a063616'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:27.289441 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52b74f338b0d491d9d0fca49f27864cb', 'request_id': 'req-9a210daa-7864-4aac-97cc-fa1a5a063616'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:27.295203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52b74f338b0d491d9d0fca49f27864cb', 'request_id': 'req-9a210daa-7864-4aac-97cc-fa1a5a063616'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:27.307705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] The token cache is being invalidate because project 52b74f338b0d491d9d0fca49f27864cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:11:27.308021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:27.308638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9a210daa-7864-4aac-97cc-fa1a5a063616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:27.310269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 971/1926] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:11:27 2022] DELETE /identity/v3/projects/52b74f338b0d491d9d0fca49f27864cb => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:27.316792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d37dd-8ee8-48ea-983c-bea62ce5828e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:27.317084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d37dd-8ee8-48ea-983c-bea62ce5828e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:27.317393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d37dd-8ee8-48ea-983c-bea62ce5828e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:27.337941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fa8d37dd-8ee8-48ea-983c-bea62ce5828e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:27.339145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fa8d37dd-8ee8-48ea-983c-bea62ce5828e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:27.396395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 956/1927] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:27.404682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7bcaa-7208-40c2-b73c-ce8fd8ff551a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:27.404967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7bcaa-7208-40c2-b73c-ce8fd8ff551a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:27.405279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac7bcaa-7208-40c2-b73c-ce8fd8ff551a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:27.425164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2ac7bcaa-7208-40c2-b73c-ce8fd8ff551a None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:27.426343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2ac7bcaa-7208-40c2-b73c-ce8fd8ff551a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:27.434882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 972/1928] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:27.439043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d37dd-8ee8-48ea-983c-bea62ce5828e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:27.449374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d37dd-8ee8-48ea-983c-bea62ce5828e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:27.454061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef99701-e523-4689-bf38-d5887069b685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:27.454657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef99701-e523-4689-bf38-d5887069b685 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:27.454935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef99701-e523-4689-bf38-d5887069b685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:27.455207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef99701-e523-4689-bf38-d5887069b685 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:27.455844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef99701-e523-4689-bf38-d5887069b685 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:11:27.456146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef99701-e523-4689-bf38-d5887069b685 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:27.457780 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef99701-e523-4689-bf38-d5887069b685 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:27.469546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 957/1929] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:11:27.473613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7bcaa-7208-40c2-b73c-ce8fd8ff551a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:27.482936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac7bcaa-7208-40c2-b73c-ce8fd8ff551a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:27.487790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55646a85-1550-45d3-ad3c-49f73973ef81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:27.488471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55646a85-1550-45d3-ad3c-49f73973ef81 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:27.488753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55646a85-1550-45d3-ad3c-49f73973ef81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:27.489024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55646a85-1550-45d3-ad3c-49f73973ef81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:27.489779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55646a85-1550-45d3-ad3c-49f73973ef81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:27.491660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55646a85-1550-45d3-ad3c-49f73973ef81 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:27.525471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 973/1930] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:11:27 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:27.526591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef99701-e523-4689-bf38-d5887069b685 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:27.536876 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef99701-e523-4689-bf38-d5887069b685 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:27.598067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fa277-4b9d-46c9-bb8e-b7105d797aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:27.598753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fa277-4b9d-46c9-bb8e-b7105d797aea admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:27.599105 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fa277-4b9d-46c9-bb8e-b7105d797aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:27.599538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fa277-4b9d-46c9-bb8e-b7105d797aea admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:27.600196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fa277-4b9d-46c9-bb8e-b7105d797aea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:27.603035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fa277-4b9d-46c9-bb8e-b7105d797aea admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:27.626847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 958/1931] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:11:27 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:11:27.631044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55646a85-1550-45d3-ad3c-49f73973ef81 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:27.642036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55646a85-1550-45d3-ad3c-49f73973ef81 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:27.646871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644984b-e0dd-420c-80ba-b9c1db2e9437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:27.647513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644984b-e0dd-420c-80ba-b9c1db2e9437 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:27.647790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644984b-e0dd-420c-80ba-b9c1db2e9437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:27.648060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644984b-e0dd-420c-80ba-b9c1db2e9437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:27.648707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644984b-e0dd-420c-80ba-b9c1db2e9437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:27.651045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b644984b-e0dd-420c-80ba-b9c1db2e9437 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:27.661417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 974/1932] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:27.666083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fa277-4b9d-46c9-bb8e-b7105d797aea admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:27.676376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fa277-4b9d-46c9-bb8e-b7105d797aea admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:27.681899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7224d-fefd-4751-87cc-532c1c84f5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:27.682492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7224d-fefd-4751-87cc-532c1c84f5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:27.682768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7224d-fefd-4751-87cc-532c1c84f5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:27.683037 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7224d-fefd-4751-87cc-532c1c84f5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:27.683676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7224d-fefd-4751-87cc-532c1c84f5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:27.685309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7224d-fefd-4751-87cc-532c1c84f5d7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:27.697832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 959/1933] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:27.701551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644984b-e0dd-420c-80ba-b9c1db2e9437 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:27.715357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644984b-e0dd-420c-80ba-b9c1db2e9437 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:27.726894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62b3c0-7624-45e3-b9ee-8a6ff82b158e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:27.728305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62b3c0-7624-45e3-b9ee-8a6ff82b158e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:27.728529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62b3c0-7624-45e3-b9ee-8a6ff82b158e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:27.728805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62b3c0-7624-45e3-b9ee-8a6ff82b158e admin admin] PATH_INFO: `/v3/projects/3136faaea7794dd9bb86ca401fdee8bb/users/6fe64cafb24648f9b69aa32c6a75fdaa/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:27.744683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62b3c0-7624-45e3-b9ee-8a6ff82b158e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3136faaea7794dd9bb86ca401fdee8bb, user_id=6fe64cafb24648f9b69aa32c6a75fdaa, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:27.747474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe62b3c0-7624-45e3-b9ee-8a6ff82b158e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:27.761669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 975/1934] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:27 2022] PUT /identity/v3/projects/3136faaea7794dd9bb86ca401fdee8bb/users/6fe64cafb24648f9b69aa32c6a75fdaa/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:11:27.772971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962adc1-1837-416c-b699-0f04d47e32ff None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:27.773521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962adc1-1837-416c-b699-0f04d47e32ff None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:27.773707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962adc1-1837-416c-b699-0f04d47e32ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:27.806600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b962adc1-1837-416c-b699-0f04d47e32ff None None] MFA Rules not processed for user `6fe64cafb24648f9b69aa32c6a75fdaa`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:27.808263 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b962adc1-1837-416c-b699-0f04d47e32ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:27.818927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:27.888117 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 960/1935] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:27.900342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62b3c0-7624-45e3-b9ee-8a6ff82b158e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:27.902077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fe62b3c0-7624-45e3-b9ee-8a6ff82b158e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:27.977954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:27.983137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe62b3c0-7624-45e3-b9ee-8a6ff82b158e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:27.987835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e343dc26-7658-443c-b84e-02908c59d4df service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '195565a7695b4d9b82e0ee76181fee2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:27.988476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343dc26-7658-443c-b84e-02908c59d4df service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:27.988753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343dc26-7658-443c-b84e-02908c59d4df service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:27.989026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e343dc26-7658-443c-b84e-02908c59d4df service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:27.998161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343dc26-7658-443c-b84e-02908c59d4df service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:27.999568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e343dc26-7658-443c-b84e-02908c59d4df service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:28.008156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 976/1936] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:37.884957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=24 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:38.004192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:46.640792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962adc1-1837-416c-b699-0f04d47e32ff None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:46.642271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b962adc1-1837-416c-b699-0f04d47e32ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:46.735077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962adc1-1837-416c-b699-0f04d47e32ff None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:46.741062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:46.741380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:46.741675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:46.741969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] PATH_INFO: `/v3/users/d523e68b249c44fa88cf48ae1a0ae7d8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:46.743697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d523e68b249c44fa88cf48ae1a0ae7d8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:46.746033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:46.799243 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd523e68b249c44fa88cf48ae1a0ae7d8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:46.799944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd523e68b249c44fa88cf48ae1a0ae7d8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:46.808414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd523e68b249c44fa88cf48ae1a0ae7d8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:46.809188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd523e68b249c44fa88cf48ae1a0ae7d8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:46.821772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd523e68b249c44fa88cf48ae1a0ae7d8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:46.835055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 961/1937] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:11:46 2022] DELETE /identity/v3/users/d523e68b249c44fa88cf48ae1a0ae7d8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:46.839896 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb60465-652e-48a1-9757-af8e73db967b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:46.840429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb60465-652e-48a1-9757-af8e73db967b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:46.840553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb60465-652e-48a1-9757-af8e73db967b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:46.860244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8eb60465-652e-48a1-9757-af8e73db967b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:46.861653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8eb60465-652e-48a1-9757-af8e73db967b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:46.926170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 977/1938] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:46.937060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:46.938617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:47.016439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b530a50-568f-422b-bca3-6251b6813046 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:47.022481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49f7de-4f9e-4541-85bb-20129ee95d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:47.025150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49f7de-4f9e-4541-85bb-20129ee95d15 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:47.025713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49f7de-4f9e-4541-85bb-20129ee95d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:47.026081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49f7de-4f9e-4541-85bb-20129ee95d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:47.040157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49f7de-4f9e-4541-85bb-20129ee95d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:47.041635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49f7de-4f9e-4541-85bb-20129ee95d15 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:47.057421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 962/1939] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:47.307997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb60465-652e-48a1-9757-af8e73db967b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:47.309534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8eb60465-652e-48a1-9757-af8e73db967b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:47.322733 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb60465-652e-48a1-9757-af8e73db967b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:47.327539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:47.328257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:47.328610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:47.329093 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] PATH_INFO: `/v3/users/5badae31e2384b919c04264f670a17a3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:47.330609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5badae31e2384b919c04264f670a17a3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:47.333311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:47.396248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5badae31e2384b919c04264f670a17a3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:47.397087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5badae31e2384b919c04264f670a17a3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:47.397821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5badae31e2384b919c04264f670a17a3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:47.411872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5badae31e2384b919c04264f670a17a3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:47.420455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5badae31e2384b919c04264f670a17a3'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:47.437090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 978/1940] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:11:47 2022] DELETE /identity/v3/users/5badae31e2384b919c04264f670a17a3 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:47.441362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b139f-2970-4857-b749-c8321ef87bda None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:47.441637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b139f-2970-4857-b749-c8321ef87bda None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:47.441915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b139f-2970-4857-b749-c8321ef87bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:47.462241 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d67b139f-2970-4857-b749-c8321ef87bda None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:47.463894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d67b139f-2970-4857-b749-c8321ef87bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:47.486339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:47.491491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:47.499465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:47.571725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 963/1941] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:47.581399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b139f-2970-4857-b749-c8321ef87bda None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:47.582949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d67b139f-2970-4857-b749-c8321ef87bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:47.620854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfec17-f3ec-431b-8fc6-a2e945316c2d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:47.627387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:47.628362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:47.628569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:47.628921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] PATH_INFO: `/v3/projects/ee5aa87e5ba34b3cae451859b7c345c2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:47.630318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee5aa87e5ba34b3cae451859b7c345c2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:47.632316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:47.657139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b139f-2970-4857-b749-c8321ef87bda None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:47.660678 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e03ddd-7951-4e39-9a87-e89c7d62dfa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:47.661566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e03ddd-7951-4e39-9a87-e89c7d62dfa6 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:47.661566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e03ddd-7951-4e39-9a87-e89c7d62dfa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:47.661859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e03ddd-7951-4e39-9a87-e89c7d62dfa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:47.663786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:47.705149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e03ddd-7951-4e39-9a87-e89c7d62dfa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:47.705149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e03ddd-7951-4e39-9a87-e89c7d62dfa6 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:47.724519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee5aa87e5ba34b3cae451859b7c345c2', 'request_id': 'req-2d469092-f509-49ae-b5f2-002df5f9b177'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:47.725586 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 964/1942] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:47.739760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee5aa87e5ba34b3cae451859b7c345c2', 'request_id': 'req-2d469092-f509-49ae-b5f2-002df5f9b177'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:47.746502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee5aa87e5ba34b3cae451859b7c345c2', 'request_id': 'req-2d469092-f509-49ae-b5f2-002df5f9b177'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:47.760917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] The token cache is being invalidate because project ee5aa87e5ba34b3cae451859b7c345c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:11:47.761530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:47.762185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:47.764027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 979/1943] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:11:47 2022] DELETE /identity/v3/projects/ee5aa87e5ba34b3cae451859b7c345c2 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:11:48.059029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e03ddd-7951-4e39-9a87-e89c7d62dfa6 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:48.060492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f9e03ddd-7951-4e39-9a87-e89c7d62dfa6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:48.084744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e03ddd-7951-4e39-9a87-e89c7d62dfa6 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:48.158200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:48.158852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:48.159133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:48.159405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] PATH_INFO: `/v3/projects/a2853b8131e448f7abe704f93a27e3dc` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:48.160541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2853b8131e448f7abe704f93a27e3dc)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:48.163340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:48.239952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2853b8131e448f7abe704f93a27e3dc', 'request_id': 'req-af6860af-4b04-4580-ac22-0929fc0be53b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:48.251129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2853b8131e448f7abe704f93a27e3dc', 'request_id': 'req-af6860af-4b04-4580-ac22-0929fc0be53b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:48.258949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2853b8131e448f7abe704f93a27e3dc', 'request_id': 'req-af6860af-4b04-4580-ac22-0929fc0be53b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:48.263911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:48.272335 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] The token cache is being invalidate because project a2853b8131e448f7abe704f93a27e3dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:11:48.272845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:48.274032 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:48.275912 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 965/1944] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:11:48 2022] DELETE /identity/v3/projects/a2853b8131e448f7abe704f93a27e3dc => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:48.279706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:48.281957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:48.302981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d469092-f509-49ae-b5f2-002df5f9b177 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:48.410478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:48.411479 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:48.412371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:48.412690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] PATH_INFO: `/v3/users/b4294aaf1f1348c98c11f4f3a1323613` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:48.413875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4294aaf1f1348c98c11f4f3a1323613)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:48.416310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:48.479642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4294aaf1f1348c98c11f4f3a1323613'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:48.480839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4294aaf1f1348c98c11f4f3a1323613'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:48.481548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4294aaf1f1348c98c11f4f3a1323613'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:48.498914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4294aaf1f1348c98c11f4f3a1323613'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:48.508337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4294aaf1f1348c98c11f4f3a1323613'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:48.528771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 980/1945] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:11:48 2022] DELETE /identity/v3/users/b4294aaf1f1348c98c11f4f3a1323613 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:11:48.859560 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:48.861533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:48.876617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6860af-4b04-4580-ac22-0929fc0be53b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:48.956854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:48.957709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e524dc-ba7e-4ade-8b1a-b607d1e1c2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:48.958350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:48.958474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e524dc-ba7e-4ade-8b1a-b607d1e1c2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:48.958684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e524dc-ba7e-4ade-8b1a-b607d1e1c2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:48.958958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e524dc-ba7e-4ade-8b1a-b607d1e1c2c7 admin admin] PATH_INFO: `/v3/users/baeb24836b6f42de9c14a061c49cc941` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:48.960359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e524dc-ba7e-4ade-8b1a-b607d1e1c2c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baeb24836b6f42de9c14a061c49cc941)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:48.963577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e524dc-ba7e-4ade-8b1a-b607d1e1c2c7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:48.974228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a5068-5e4d-4443-b4cb-3f5a43f46555 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:48.980388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:48.980825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:48.981153 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:48.981439 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] PATH_INFO: `/v3/projects/0a2dd772b1624f23ab1716ca79e1e70a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:48.983061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a2dd772b1624f23ab1716ca79e1e70a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:48.984975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:49.006503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:49.036015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d6e524dc-ba7e-4ade-8b1a-b607d1e1c2c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baeb24836b6f42de9c14a061c49cc941'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:49.036778 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d6e524dc-ba7e-4ade-8b1a-b607d1e1c2c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baeb24836b6f42de9c14a061c49cc941'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:49.046412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d6e524dc-ba7e-4ade-8b1a-b607d1e1c2c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baeb24836b6f42de9c14a061c49cc941'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:49.047454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d6e524dc-ba7e-4ade-8b1a-b607d1e1c2c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baeb24836b6f42de9c14a061c49cc941'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:49.049062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a2dd772b1624f23ab1716ca79e1e70a', 'request_id': 'req-d367dc88-94be-4aba-8af9-e3e19a2310ef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:49.056814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a2dd772b1624f23ab1716ca79e1e70a', 'request_id': 'req-d367dc88-94be-4aba-8af9-e3e19a2310ef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:49.064527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a2dd772b1624f23ab1716ca79e1e70a', 'request_id': 'req-d367dc88-94be-4aba-8af9-e3e19a2310ef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:49.064904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d6e524dc-ba7e-4ade-8b1a-b607d1e1c2c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baeb24836b6f42de9c14a061c49cc941'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:49.079760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] The token cache is being invalidate because project 0a2dd772b1624f23ab1716ca79e1e70a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:11:49.080272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:49.080983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d367dc88-94be-4aba-8af9-e3e19a2310ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:49.082126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 966/1946] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:11:48 2022] DELETE /identity/v3/users/baeb24836b6f42de9c14a061c49cc941 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:49.082930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 981/1947] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:11:48 2022] DELETE /identity/v3/projects/0a2dd772b1624f23ab1716ca79e1e70a => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:11:49.087887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3a170-5918-479a-8fa6-d33214afdeb1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.088435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3a170-5918-479a-8fa6-d33214afdeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.088720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3a170-5918-479a-8fa6-d33214afdeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.089707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920301d-3c97-44f2-9d53-06066862c973 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.090239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920301d-3c97-44f2-9d53-06066862c973 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.090614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1920301d-3c97-44f2-9d53-06066862c973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.109870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8ad3a170-5918-479a-8fa6-d33214afdeb1 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:49.111452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8ad3a170-5918-479a-8fa6-d33214afdeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:49.116922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1920301d-3c97-44f2-9d53-06066862c973 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:49.117751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1920301d-3c97-44f2-9d53-06066862c973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:49.184632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 967/1948] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:11:49.192149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf299-621c-4b26-9ce5-e433ad680b9e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.192441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf299-621c-4b26-9ce5-e433ad680b9e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.192715 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afbf299-621c-4b26-9ce5-e433ad680b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.196442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 982/1949] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:49.210175 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920301d-3c97-44f2-9d53-06066862c973 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:49.211702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1920301d-3c97-44f2-9d53-06066862c973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:49.214188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2afbf299-621c-4b26-9ce5-e433ad680b9e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:49.215365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2afbf299-621c-4b26-9ce5-e433ad680b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:49.224760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 968/1950] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:11:49.229947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbf299-621c-4b26-9ce5-e433ad680b9e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:49.243297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afbf299-621c-4b26-9ce5-e433ad680b9e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:49.248838 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dba3a7-5dcd-4d95-aa76-d030a7909fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:49.249599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dba3a7-5dcd-4d95-aa76-d030a7909fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.249942 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dba3a7-5dcd-4d95-aa76-d030a7909fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.250313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dba3a7-5dcd-4d95-aa76-d030a7909fe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.251004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dba3a7-5dcd-4d95-aa76-d030a7909fe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:11:49.251374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dba3a7-5dcd-4d95-aa76-d030a7909fe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:49.253365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dba3a7-5dcd-4d95-aa76-d030a7909fe0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:49.256645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:49.268536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 969/1951] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:11:49.272713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dba3a7-5dcd-4d95-aa76-d030a7909fe0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:49.286876 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dba3a7-5dcd-4d95-aa76-d030a7909fe0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:49.287885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1920301d-3c97-44f2-9d53-06066862c973 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:49.292762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ed131-478a-4d25-a6d2-fd9e9abbc1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:49.293413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ed131-478a-4d25-a6d2-fd9e9abbc1da service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.293694 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ed131-478a-4d25-a6d2-fd9e9abbc1da service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.293970 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ed131-478a-4d25-a6d2-fd9e9abbc1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.294326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6a8dd-5cf6-4383-b638-fda99da6b31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:49.295040 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6a8dd-5cf6-4383-b638-fda99da6b31c admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.295591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6a8dd-5cf6-4383-b638-fda99da6b31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.296074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6a8dd-5cf6-4383-b638-fda99da6b31c admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.297329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6a8dd-5cf6-4383-b638-fda99da6b31c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:49.299738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b6a8dd-5cf6-4383-b638-fda99da6b31c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:49.308506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ed131-478a-4d25-a6d2-fd9e9abbc1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:49.310192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ed131-478a-4d25-a6d2-fd9e9abbc1da service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:49.342052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 983/1952] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:49.369627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 970/1953] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:11:49 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:49.374243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ed131-478a-4d25-a6d2-fd9e9abbc1da service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:49.400794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ed131-478a-4d25-a6d2-fd9e9abbc1da service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:49.535737 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:49.560482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0f209-4a41-4bb5-bf54-159a64dbfdd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:49.561328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0f209-4a41-4bb5-bf54-159a64dbfdd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.573703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0f209-4a41-4bb5-bf54-159a64dbfdd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.574049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0f209-4a41-4bb5-bf54-159a64dbfdd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.574708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0f209-4a41-4bb5-bf54-159a64dbfdd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:49.587290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0f209-4a41-4bb5-bf54-159a64dbfdd5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:49.629853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 984/1954] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:11:49 2022] POST /identity/v3/users => generated 296 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:49.633701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6a8dd-5cf6-4383-b638-fda99da6b31c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:49.648169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b6a8dd-5cf6-4383-b638-fda99da6b31c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:49.657542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bad9ee-c341-4092-995b-dbf9caf21aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:49.657869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bad9ee-c341-4092-995b-dbf9caf21aac admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.658148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bad9ee-c341-4092-995b-dbf9caf21aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.660404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bad9ee-c341-4092-995b-dbf9caf21aac admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.661144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bad9ee-c341-4092-995b-dbf9caf21aac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:49.666500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bad9ee-c341-4092-995b-dbf9caf21aac admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:49.689150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 971/1955] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:49.693746 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0f209-4a41-4bb5-bf54-159a64dbfdd5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:49.707921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0f209-4a41-4bb5-bf54-159a64dbfdd5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:49.727650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d7119-ca55-4319-a6a9-8a10a00408d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:49.727650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d7119-ca55-4319-a6a9-8a10a00408d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.727650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d7119-ca55-4319-a6a9-8a10a00408d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.727650 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d7119-ca55-4319-a6a9-8a10a00408d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.728186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d7119-ca55-4319-a6a9-8a10a00408d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:49.728186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d7119-ca55-4319-a6a9-8a10a00408d4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:49.733323 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 985/1956] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:11:49.738645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bad9ee-c341-4092-995b-dbf9caf21aac admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:49.761844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bad9ee-c341-4092-995b-dbf9caf21aac admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:49.768605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8698b2-15a5-4d45-9aac-e0c97fbf703b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:49.769744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8698b2-15a5-4d45-9aac-e0c97fbf703b admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.770146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8698b2-15a5-4d45-9aac-e0c97fbf703b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.770659 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8698b2-15a5-4d45-9aac-e0c97fbf703b admin admin] PATH_INFO: `/v3/projects/ec27f2ce936e40e8b5ac4ea1d21d6609/users/1fa2336f41bd45d594cb19ee2a6c5f34/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.813052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8698b2-15a5-4d45-9aac-e0c97fbf703b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec27f2ce936e40e8b5ac4ea1d21d6609, user_id=1fa2336f41bd45d594cb19ee2a6c5f34, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:49.816039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8698b2-15a5-4d45-9aac-e0c97fbf703b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:49.836662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 972/1957] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:49 2022] PUT /identity/v3/projects/ec27f2ce936e40e8b5ac4ea1d21d6609/users/1fa2336f41bd45d594cb19ee2a6c5f34/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:49.848888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6e237-e7d7-4130-95ef-134fe5030aa8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.849196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6e237-e7d7-4130-95ef-134fe5030aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.849474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c6e237-e7d7-4130-95ef-134fe5030aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.895231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8698b2-15a5-4d45-9aac-e0c97fbf703b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:49.897408 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4d8698b2-15a5-4d45-9aac-e0c97fbf703b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:49.903135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-95c6e237-e7d7-4130-95ef-134fe5030aa8 None None] MFA Rules not processed for user `1fa2336f41bd45d594cb19ee2a6c5f34`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:49.905052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-95c6e237-e7d7-4130-95ef-134fe5030aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:49.989523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8698b2-15a5-4d45-9aac-e0c97fbf703b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:49.994453 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 986/1958] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:49.995344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:49.996095 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:49.996373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:49.996823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] PATH_INFO: `/v3/projects/3090011abe2f47fc803b45e9aa84c70c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:49.998436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3090011abe2f47fc803b45e9aa84c70c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:49.998789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6e237-e7d7-4130-95ef-134fe5030aa8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:50.000736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:50.010544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c6e237-e7d7-4130-95ef-134fe5030aa8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:50.015399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0cb8d-5c2e-44b4-b410-93736c13a663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:50.016536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0cb8d-5c2e-44b4-b410-93736c13a663 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:50.016536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0cb8d-5c2e-44b4-b410-93736c13a663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:50.016795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0cb8d-5c2e-44b4-b410-93736c13a663 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:50.017649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0cb8d-5c2e-44b4-b410-93736c13a663 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:50.020069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0cb8d-5c2e-44b4-b410-93736c13a663 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:50.053814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 987/1959] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:11:49 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:11:50.059145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0cb8d-5c2e-44b4-b410-93736c13a663 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:50.074162 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3090011abe2f47fc803b45e9aa84c70c', 'request_id': 'req-cbbde2e8-51e5-485e-9310-612dab025865'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:50.084030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3090011abe2f47fc803b45e9aa84c70c', 'request_id': 'req-cbbde2e8-51e5-485e-9310-612dab025865'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:50.092675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3090011abe2f47fc803b45e9aa84c70c', 'request_id': 'req-cbbde2e8-51e5-485e-9310-612dab025865'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:50.107879 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] The token cache is being invalidate because project 3090011abe2f47fc803b45e9aa84c70c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:11:50.108185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:50.108965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:50.110716 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 973/1960] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:11:49 2022] DELETE /identity/v3/projects/3090011abe2f47fc803b45e9aa84c70c => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:50.114391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:50.115737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:50.136624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0cb8d-5c2e-44b4-b410-93736c13a663 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:50.142074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81211d-416a-4fc9-a451-b98d6fdf84df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:50.143167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81211d-416a-4fc9-a451-b98d6fdf84df admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:50.143503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81211d-416a-4fc9-a451-b98d6fdf84df admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:50.143784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81211d-416a-4fc9-a451-b98d6fdf84df admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:50.144876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81211d-416a-4fc9-a451-b98d6fdf84df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:50.147787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd81211d-416a-4fc9-a451-b98d6fdf84df admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:50.149836 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:50.181859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 988/1961] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:11:50 2022] POST /identity/v3/users => generated 302 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:50.183087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81211d-416a-4fc9-a451-b98d6fdf84df admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:50.185020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cd81211d-416a-4fc9-a451-b98d6fdf84df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:50.197406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbde2e8-51e5-485e-9310-612dab025865 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:50.201682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd81211d-416a-4fc9-a451-b98d6fdf84df admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:50.202617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:50.203269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:50.203550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:50.203852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] PATH_INFO: `/v3/users/afe831c11a564b04a1ec5f3d297547f0` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:50.205050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe831c11a564b04a1ec5f3d297547f0)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:50.207083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:50.212571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d6cd5-2961-44f1-8aca-e7c6dc16d1cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:50.213375 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d6cd5-2961-44f1-8aca-e7c6dc16d1cd admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:50.213759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d6cd5-2961-44f1-8aca-e7c6dc16d1cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:50.214937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d6cd5-2961-44f1-8aca-e7c6dc16d1cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:50.215712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d6cd5-2961-44f1-8aca-e7c6dc16d1cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:50.217482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064d6cd5-2961-44f1-8aca-e7c6dc16d1cd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:50.230697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 989/1962] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:50.235603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d6cd5-2961-44f1-8aca-e7c6dc16d1cd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:50.259508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d6cd5-2961-44f1-8aca-e7c6dc16d1cd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:50.266423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1af84f-960f-4590-898f-2e3c82978e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:50.267233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1af84f-960f-4590-898f-2e3c82978e86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:50.267670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1af84f-960f-4590-898f-2e3c82978e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:50.268001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1af84f-960f-4590-898f-2e3c82978e86 admin admin] PATH_INFO: `/v3/projects/4399a83ae2d4406491d44a5ba881608f/users/c3432d2968c544f2a4a9254de89f41db/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:50.288959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:50.318320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1af84f-960f-4590-898f-2e3c82978e86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4399a83ae2d4406491d44a5ba881608f, user_id=c3432d2968c544f2a4a9254de89f41db, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:50.328566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1af84f-960f-4590-898f-2e3c82978e86 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:50.337943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe831c11a564b04a1ec5f3d297547f0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:50.338931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe831c11a564b04a1ec5f3d297547f0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:50.353652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 990/1963] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:50 2022] PUT /identity/v3/projects/4399a83ae2d4406491d44a5ba881608f/users/c3432d2968c544f2a4a9254de89f41db/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:50.357905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1af84f-960f-4590-898f-2e3c82978e86 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:50.359190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe831c11a564b04a1ec5f3d297547f0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:50.360257 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe831c11a564b04a1ec5f3d297547f0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:50.361046 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0c1af84f-960f-4590-898f-2e3c82978e86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:50.379263 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe831c11a564b04a1ec5f3d297547f0'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:50.400044 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 974/1964] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:11:50 2022] DELETE /identity/v3/users/afe831c11a564b04a1ec5f3d297547f0 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:11:50.450946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1af84f-960f-4590-898f-2e3c82978e86 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:50.565576 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0f81e3-afdc-4b29-aa98-fb473439a853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:50.566265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f81e3-afdc-4b29-aa98-fb473439a853 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:50.566542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f81e3-afdc-4b29-aa98-fb473439a853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:50.566812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0f81e3-afdc-4b29-aa98-fb473439a853 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:50.567483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f81e3-afdc-4b29-aa98-fb473439a853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:50.569154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0f81e3-afdc-4b29-aa98-fb473439a853 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:50.595950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 991/1965] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:11:50.598222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:50.610995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b819ca-65de-4b68-be8c-961a41b4013f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:50.618396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bbd524-df73-4dbd-a870-4c2b80062a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:50.619167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbd524-df73-4dbd-a870-4c2b80062a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:50.619452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbd524-df73-4dbd-a870-4c2b80062a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:50.619749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bbd524-df73-4dbd-a870-4c2b80062a20 admin admin] PATH_INFO: `/v3/projects/4399a83ae2d4406491d44a5ba881608f/users/c3432d2968c544f2a4a9254de89f41db/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:50.621860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bbd524-df73-4dbd-a870-4c2b80062a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4399a83ae2d4406491d44a5ba881608f, user_id=c3432d2968c544f2a4a9254de89f41db, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:50.624848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bbd524-df73-4dbd-a870-4c2b80062a20 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:50.646264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 975/1966] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:50 2022] PUT /identity/v3/projects/4399a83ae2d4406491d44a5ba881608f/users/c3432d2968c544f2a4a9254de89f41db/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:50.658172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf16e4-add0-48a5-a417-afb70327b109 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:50.658342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf16e4-add0-48a5-a417-afb70327b109 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:50.658550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddf16e4-add0-48a5-a417-afb70327b109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:50.692127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5ddf16e4-add0-48a5-a417-afb70327b109 None None] MFA Rules not processed for user `c3432d2968c544f2a4a9254de89f41db`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:50.693887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5ddf16e4-add0-48a5-a417-afb70327b109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:50.800116 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bbd524-df73-4dbd-a870-4c2b80062a20 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:50.805770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a4bbd524-df73-4dbd-a870-4c2b80062a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:50.844547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 992/1967] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:50 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:11:50.909077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf16e4-add0-48a5-a417-afb70327b109 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:50.911204 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5ddf16e4-add0-48a5-a417-afb70327b109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:50.937552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:50.943888 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bbd524-df73-4dbd-a870-4c2b80062a20 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:50.952927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:50.955637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:50.956070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:50.956504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] PATH_INFO: `/v3/projects/b6ddda7779a144a2a51e4905f4329ef4` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:50.958443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6ddda7779a144a2a51e4905f4329ef4)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:50.960894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:50.986507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddf16e4-add0-48a5-a417-afb70327b109 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:50.991554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64700c59-54d5-4908-b3fe-75dee35590fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:50.992481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64700c59-54d5-4908-b3fe-75dee35590fb service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:50.992885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64700c59-54d5-4908-b3fe-75dee35590fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:50.993291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64700c59-54d5-4908-b3fe-75dee35590fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.007484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64700c59-54d5-4908-b3fe-75dee35590fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:51.010538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64700c59-54d5-4908-b3fe-75dee35590fb service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:51.030273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:51.032390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 993/1968] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:50 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:51.045282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6ddda7779a144a2a51e4905f4329ef4', 'request_id': 'req-a2d8fd9d-df82-4fda-b750-11cb319d4f73'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:51.067159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6ddda7779a144a2a51e4905f4329ef4', 'request_id': 'req-a2d8fd9d-df82-4fda-b750-11cb319d4f73'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:51.075830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6ddda7779a144a2a51e4905f4329ef4', 'request_id': 'req-a2d8fd9d-df82-4fda-b750-11cb319d4f73'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:51.092249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] The token cache is being invalidate because project b6ddda7779a144a2a51e4905f4329ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:11:51.092249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:51.092249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:51.096513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 976/1969] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:11:50 2022] DELETE /identity/v3/projects/b6ddda7779a144a2a51e4905f4329ef4 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:51.100279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddff9df-e5bc-43f7-93d0-3a94dd0d41e4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:51.101420 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddff9df-e5bc-43f7-93d0-3a94dd0d41e4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:51.101750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddff9df-e5bc-43f7-93d0-3a94dd0d41e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.134932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1ddff9df-e5bc-43f7-93d0-3a94dd0d41e4 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:51.136813 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1ddff9df-e5bc-43f7-93d0-3a94dd0d41e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:51.199675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:51.201134 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:51.249985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 994/1970] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:51.257446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb0fcc-84fd-4a86-865f-02a711eb11da None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:51.258556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb0fcc-84fd-4a86-865f-02a711eb11da None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:51.258966 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb0fcc-84fd-4a86-865f-02a711eb11da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.279774 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8bdb0fcc-84fd-4a86-865f-02a711eb11da None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:51.281074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8bdb0fcc-84fd-4a86-865f-02a711eb11da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:51.282104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8fd9d-df82-4fda-b750-11cb319d4f73 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:51.285794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5202a0-ab6d-4b5a-a3a7-9072baf55ef4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:51.286554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5202a0-ab6d-4b5a-a3a7-9072baf55ef4 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:51.286894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5202a0-ab6d-4b5a-a3a7-9072baf55ef4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:51.287189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5202a0-ab6d-4b5a-a3a7-9072baf55ef4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.289286 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9e5202a0-ab6d-4b5a-a3a7-9072baf55ef4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:51.292333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 995/1971] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:11:51.296502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb0fcc-84fd-4a86-865f-02a711eb11da None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:51.306431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb0fcc-84fd-4a86-865f-02a711eb11da None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:51.311825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9469ff5-f08b-4daa-a38f-0e0781416335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:51.312502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9469ff5-f08b-4daa-a38f-0e0781416335 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:51.312846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9469ff5-f08b-4daa-a38f-0e0781416335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:51.313226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9469ff5-f08b-4daa-a38f-0e0781416335 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.313942 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9469ff5-f08b-4daa-a38f-0e0781416335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:11:51.314312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9469ff5-f08b-4daa-a38f-0e0781416335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:51.315969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9469ff5-f08b-4daa-a38f-0e0781416335 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:51.331436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 996/1972] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:11:51.332299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9469ff5-f08b-4daa-a38f-0e0781416335 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:51.345669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9469ff5-f08b-4daa-a38f-0e0781416335 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:51.353582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fc880-7f99-49c9-8493-f906b466ffe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:51.354127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc880-7f99-49c9-8493-f906b466ffe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:51.354409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc880-7f99-49c9-8493-f906b466ffe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:51.354686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fc880-7f99-49c9-8493-f906b466ffe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.355664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fc880-7f99-49c9-8493-f906b466ffe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:51.358006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fc880-7f99-49c9-8493-f906b466ffe7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:51.358521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5202a0-ab6d-4b5a-a3a7-9072baf55ef4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:51.359950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5202a0-ab6d-4b5a-a3a7-9072baf55ef4 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:51.371613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 977/1973] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:51.393779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 997/1974] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:11:51 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:51.398633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5202a0-ab6d-4b5a-a3a7-9072baf55ef4 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:51.410647 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5202a0-ab6d-4b5a-a3a7-9072baf55ef4 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:51.487875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfafca5-2383-4d50-9da1-a69146d0365d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:51.487875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfafca5-2383-4d50-9da1-a69146d0365d admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:51.487875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfafca5-2383-4d50-9da1-a69146d0365d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:51.488148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfafca5-2383-4d50-9da1-a69146d0365d admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.488656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfafca5-2383-4d50-9da1-a69146d0365d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:51.490632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfafca5-2383-4d50-9da1-a69146d0365d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:51.516025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 978/1975] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:11:51 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:51.520951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fc880-7f99-49c9-8493-f906b466ffe7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:51.534978 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fc880-7f99-49c9-8493-f906b466ffe7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:51.541160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23154d7b-84d3-4152-9834-8d6d81281add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:51.541923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23154d7b-84d3-4152-9834-8d6d81281add admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:51.542206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23154d7b-84d3-4152-9834-8d6d81281add admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:51.542542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23154d7b-84d3-4152-9834-8d6d81281add admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.543461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23154d7b-84d3-4152-9834-8d6d81281add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:51.545178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23154d7b-84d3-4152-9834-8d6d81281add admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:51.560008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 998/1976] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:51.564362 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfafca5-2383-4d50-9da1-a69146d0365d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:51.577139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfafca5-2383-4d50-9da1-a69146d0365d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:51.582611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e8b233-2713-4c0b-9e88-4b361ca0f3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:51.583501 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8b233-2713-4c0b-9e88-4b361ca0f3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:51.583784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8b233-2713-4c0b-9e88-4b361ca0f3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:51.584070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e8b233-2713-4c0b-9e88-4b361ca0f3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.584883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8b233-2713-4c0b-9e88-4b361ca0f3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:51.586898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e8b233-2713-4c0b-9e88-4b361ca0f3b8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:51.590436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:51.612689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 979/1977] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:51.612852 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23154d7b-84d3-4152-9834-8d6d81281add admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:51.623765 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23154d7b-84d3-4152-9834-8d6d81281add admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:51.631272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c54ae8-4ff7-4d72-b842-f6ccf8feb7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:51.633807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c54ae8-4ff7-4d72-b842-f6ccf8feb7d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:51.633979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c54ae8-4ff7-4d72-b842-f6ccf8feb7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:51.635206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c54ae8-4ff7-4d72-b842-f6ccf8feb7d4 admin admin] PATH_INFO: `/v3/projects/f71a3207ec8f464792e63f7d78e5c946/users/747337ac2f6f46f3b26f18c4c8ec22bd/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.654101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c54ae8-4ff7-4d72-b842-f6ccf8feb7d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f71a3207ec8f464792e63f7d78e5c946, user_id=747337ac2f6f46f3b26f18c4c8ec22bd, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:51.657039 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c54ae8-4ff7-4d72-b842-f6ccf8feb7d4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:51.675400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 999/1978] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:51 2022] PUT /identity/v3/projects/f71a3207ec8f464792e63f7d78e5c946/users/747337ac2f6f46f3b26f18c4c8ec22bd/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:11:51.688081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c48502-8282-475c-8074-e2f7e33da0e0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:51.689424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c48502-8282-475c-8074-e2f7e33da0e0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:51.689721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c48502-8282-475c-8074-e2f7e33da0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.725751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c6c48502-8282-475c-8074-e2f7e33da0e0 None None] MFA Rules not processed for user `747337ac2f6f46f3b26f18c4c8ec22bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:51.728723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c6c48502-8282-475c-8074-e2f7e33da0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:51.824148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 980/1979] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:51.828339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c54ae8-4ff7-4d72-b842-f6ccf8feb7d4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:51.877576 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:51.896194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c54ae8-4ff7-4d72-b842-f6ccf8feb7d4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:51.901736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20080194-124a-426d-bacd-0c7e99618ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:51.902351 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20080194-124a-426d-bacd-0c7e99618ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:51.902661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20080194-124a-426d-bacd-0c7e99618ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:51.902931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20080194-124a-426d-bacd-0c7e99618ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:51.903571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20080194-124a-426d-bacd-0c7e99618ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:51.905504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20080194-124a-426d-bacd-0c7e99618ad1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:51.938852 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1000/1980] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:11:51 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:51.942930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c48502-8282-475c-8074-e2f7e33da0e0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:51.957447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c48502-8282-475c-8074-e2f7e33da0e0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:52.031363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80d5f1-5115-49de-8007-fefc4b9882b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:52.031971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d5f1-5115-49de-8007-fefc4b9882b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:52.032246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d5f1-5115-49de-8007-fefc4b9882b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:52.032515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e80d5f1-5115-49de-8007-fefc4b9882b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:52.033191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80d5f1-5115-49de-8007-fefc4b9882b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:52.035183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e80d5f1-5115-49de-8007-fefc4b9882b1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:52.076060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 981/1981] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:11:51 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:52.076263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20080194-124a-426d-bacd-0c7e99618ad1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:52.081250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20080194-124a-426d-bacd-0c7e99618ad1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:52.088376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25832e1-606c-4ffb-9b9d-4335e82c2208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:52.088992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25832e1-606c-4ffb-9b9d-4335e82c2208 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:52.089293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25832e1-606c-4ffb-9b9d-4335e82c2208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:52.089568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25832e1-606c-4ffb-9b9d-4335e82c2208 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:52.090211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25832e1-606c-4ffb-9b9d-4335e82c2208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:52.091792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25832e1-606c-4ffb-9b9d-4335e82c2208 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:52.102960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1001/1982] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:52.107064 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80d5f1-5115-49de-8007-fefc4b9882b1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:52.117433 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e80d5f1-5115-49de-8007-fefc4b9882b1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:52.122723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aba7c-fec0-401c-9346-8928a5a6fe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:52.123383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aba7c-fec0-401c-9346-8928a5a6fe07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:52.123662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aba7c-fec0-401c-9346-8928a5a6fe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:52.123935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1aba7c-fec0-401c-9346-8928a5a6fe07 admin admin] PATH_INFO: `/v3/projects/25941b21709344cfbfe794b6d563906d/users/8704bba52e754068a17957ac57800dd6/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:52.140339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1aba7c-fec0-401c-9346-8928a5a6fe07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25941b21709344cfbfe794b6d563906d, user_id=8704bba52e754068a17957ac57800dd6, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:52.142945 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1aba7c-fec0-401c-9346-8928a5a6fe07 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:52.156711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 982/1983] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:52 2022] PUT /identity/v3/projects/25941b21709344cfbfe794b6d563906d/users/8704bba52e754068a17957ac57800dd6/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:11:52.160912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25832e1-606c-4ffb-9b9d-4335e82c2208 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:52.171752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25832e1-606c-4ffb-9b9d-4335e82c2208 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:52.254597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d233e7b-dbb9-42ff-bdc9-9167041140f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:52.255229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d233e7b-dbb9-42ff-bdc9-9167041140f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:52.255529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d233e7b-dbb9-42ff-bdc9-9167041140f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:52.255850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d233e7b-dbb9-42ff-bdc9-9167041140f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:52.256504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d233e7b-dbb9-42ff-bdc9-9167041140f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:52.259260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d233e7b-dbb9-42ff-bdc9-9167041140f2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:52.272107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1002/1984] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:52.276560 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aba7c-fec0-401c-9346-8928a5a6fe07 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:52.287070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1aba7c-fec0-401c-9346-8928a5a6fe07 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:52.291778 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d398b51-b83e-4ddf-8f0a-7e1c2a82d027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:52.292432 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d398b51-b83e-4ddf-8f0a-7e1c2a82d027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:52.292709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d398b51-b83e-4ddf-8f0a-7e1c2a82d027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:52.292979 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d398b51-b83e-4ddf-8f0a-7e1c2a82d027 admin admin] PATH_INFO: `/v3/projects/25941b21709344cfbfe794b6d563906d/users/8704bba52e754068a17957ac57800dd6/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:52.294917 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d398b51-b83e-4ddf-8f0a-7e1c2a82d027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25941b21709344cfbfe794b6d563906d, user_id=8704bba52e754068a17957ac57800dd6, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:52.297565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d398b51-b83e-4ddf-8f0a-7e1c2a82d027 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:52.309075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 983/1985] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:52 2022] PUT /identity/v3/projects/25941b21709344cfbfe794b6d563906d/users/8704bba52e754068a17957ac57800dd6/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:52.319121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b2fb1-1e40-4d57-a37d-77ef48495e5c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:52.319626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b2fb1-1e40-4d57-a37d-77ef48495e5c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:52.319720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b2fb1-1e40-4d57-a37d-77ef48495e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:52.350064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-339b2fb1-1e40-4d57-a37d-77ef48495e5c None None] MFA Rules not processed for user `8704bba52e754068a17957ac57800dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:52.351729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-339b2fb1-1e40-4d57-a37d-77ef48495e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:52.426564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1003/1986] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:52.450551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d398b51-b83e-4ddf-8f0a-7e1c2a82d027 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:52.502075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:52.517846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d398b51-b83e-4ddf-8f0a-7e1c2a82d027 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:52.521768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7828bea-781a-4d41-bf5e-a6ef1df3ff9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:52.522742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7828bea-781a-4d41-bf5e-a6ef1df3ff9f service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:52.523065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7828bea-781a-4d41-bf5e-a6ef1df3ff9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:52.523898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7828bea-781a-4d41-bf5e-a6ef1df3ff9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:52.533011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7828bea-781a-4d41-bf5e-a6ef1df3ff9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:52.534459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7828bea-781a-4d41-bf5e-a6ef1df3ff9f service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:52.544141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 984/1987] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:52.622772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b2fb1-1e40-4d57-a37d-77ef48495e5c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:52.632350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b2fb1-1e40-4d57-a37d-77ef48495e5c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:52.636874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a4ca21-e834-49d8-88f1-ee58903e5973 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:52.637507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4ca21-e834-49d8-88f1-ee58903e5973 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:52.637795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4ca21-e834-49d8-88f1-ee58903e5973 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:52.638071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a4ca21-e834-49d8-88f1-ee58903e5973 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:52.680856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:52.704334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4ca21-e834-49d8-88f1-ee58903e5973 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:52.705908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a4ca21-e834-49d8-88f1-ee58903e5973 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:52.719403 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1004/1988] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:54.899772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7828bea-781a-4d41-bf5e-a6ef1df3ff9f service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:54.901238 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b7828bea-781a-4d41-bf5e-a6ef1df3ff9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:54.965857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7828bea-781a-4d41-bf5e-a6ef1df3ff9f service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:54.970313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:54.970926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:54.971202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:54.971474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] PATH_INFO: `/v3/users/1fa2336f41bd45d594cb19ee2a6c5f34` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:54.972580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fa2336f41bd45d594cb19ee2a6c5f34)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:54.974518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:55.020061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fa2336f41bd45d594cb19ee2a6c5f34'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.020724 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fa2336f41bd45d594cb19ee2a6c5f34'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.028187 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fa2336f41bd45d594cb19ee2a6c5f34'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.028828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fa2336f41bd45d594cb19ee2a6c5f34'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.040875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fa2336f41bd45d594cb19ee2a6c5f34'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.053608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 985/1989] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:11:54 2022] DELETE /identity/v3/users/1fa2336f41bd45d594cb19ee2a6c5f34 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:55.058712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045548b0-0dad-4e77-ba25-f8cebddf0b6d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:55.059164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045548b0-0dad-4e77-ba25-f8cebddf0b6d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:55.059164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045548b0-0dad-4e77-ba25-f8cebddf0b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:55.078483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-045548b0-0dad-4e77-ba25-f8cebddf0b6d None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:55.079684 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-045548b0-0dad-4e77-ba25-f8cebddf0b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:55.136960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1005/1990] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:55.146895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:55.149449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:55.205401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42c2af-d6c1-4087-8614-183958409bf1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:55.208931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e1459-ea8c-413f-94f7-11da12623ed0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:55.209554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e1459-ea8c-413f-94f7-11da12623ed0 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:55.209829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e1459-ea8c-413f-94f7-11da12623ed0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:55.210096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e1459-ea8c-413f-94f7-11da12623ed0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:55.219555 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e1459-ea8c-413f-94f7-11da12623ed0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:55.220911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e1459-ea8c-413f-94f7-11da12623ed0 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:55.230945 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 986/1991] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:55.504601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045548b0-0dad-4e77-ba25-f8cebddf0b6d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:55.506054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-045548b0-0dad-4e77-ba25-f8cebddf0b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:55.520614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045548b0-0dad-4e77-ba25-f8cebddf0b6d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:55.526378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:55.527021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:55.527297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:55.527571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] PATH_INFO: `/v3/projects/ec27f2ce936e40e8b5ac4ea1d21d6609` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:55.528716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec27f2ce936e40e8b5ac4ea1d21d6609)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:55.530737 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:55.583440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec27f2ce936e40e8b5ac4ea1d21d6609', 'request_id': 'req-1dc0b120-1d8d-471c-8d99-bc9ff232b735'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.590828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec27f2ce936e40e8b5ac4ea1d21d6609', 'request_id': 'req-1dc0b120-1d8d-471c-8d99-bc9ff232b735'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.597399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec27f2ce936e40e8b5ac4ea1d21d6609', 'request_id': 'req-1dc0b120-1d8d-471c-8d99-bc9ff232b735'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.611995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] The token cache is being invalidate because project ec27f2ce936e40e8b5ac4ea1d21d6609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:11:55.612287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.613099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.614945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1006/1992] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:11:55 2022] DELETE /identity/v3/projects/ec27f2ce936e40e8b5ac4ea1d21d6609 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:55.619108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e1459-ea8c-413f-94f7-11da12623ed0 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:55.625671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:55.629775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e1459-ea8c-413f-94f7-11da12623ed0 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:55.694035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8863bd-fa79-4460-a8af-09f21ba7583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:55.695217 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8863bd-fa79-4460-a8af-09f21ba7583f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:55.695498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8863bd-fa79-4460-a8af-09f21ba7583f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:55.695769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8863bd-fa79-4460-a8af-09f21ba7583f admin admin] PATH_INFO: `/v3/users/c3432d2968c544f2a4a9254de89f41db` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:55.696962 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8863bd-fa79-4460-a8af-09f21ba7583f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3432d2968c544f2a4a9254de89f41db)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:55.698949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8863bd-fa79-4460-a8af-09f21ba7583f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:55.749285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2a8863bd-fa79-4460-a8af-09f21ba7583f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3432d2968c544f2a4a9254de89f41db'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.749859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2a8863bd-fa79-4460-a8af-09f21ba7583f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3432d2968c544f2a4a9254de89f41db'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.756961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2a8863bd-fa79-4460-a8af-09f21ba7583f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3432d2968c544f2a4a9254de89f41db'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.758334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2a8863bd-fa79-4460-a8af-09f21ba7583f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3432d2968c544f2a4a9254de89f41db'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.771755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2a8863bd-fa79-4460-a8af-09f21ba7583f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3432d2968c544f2a4a9254de89f41db'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:55.786449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 987/1993] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:11:55 2022] DELETE /identity/v3/users/c3432d2968c544f2a4a9254de89f41db => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:56.126239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:56.127904 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:56.141504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc0b120-1d8d-471c-8d99-bc9ff232b735 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:56.207946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:56.208577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:56.208855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:56.209154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] PATH_INFO: `/v3/projects/4399a83ae2d4406491d44a5ba881608f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:56.210423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4399a83ae2d4406491d44a5ba881608f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:56.212893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:56.216654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:56.261258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4399a83ae2d4406491d44a5ba881608f', 'request_id': 'req-900c6a19-d17a-48e8-8f0f-113a74795ba7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:56.268217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4399a83ae2d4406491d44a5ba881608f', 'request_id': 'req-900c6a19-d17a-48e8-8f0f-113a74795ba7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:56.274630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4399a83ae2d4406491d44a5ba881608f', 'request_id': 'req-900c6a19-d17a-48e8-8f0f-113a74795ba7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:56.290371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] The token cache is being invalidate because project 4399a83ae2d4406491d44a5ba881608f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:11:56.290371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:56.290602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-900c6a19-d17a-48e8-8f0f-113a74795ba7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:11:56.292041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1007/1994] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:11:56 2022] DELETE /identity/v3/projects/4399a83ae2d4406491d44a5ba881608f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:56.296929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac243b-3b49-4c3d-a50e-6397b29dc652 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:56.297541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac243b-3b49-4c3d-a50e-6397b29dc652 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:56.297541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac243b-3b49-4c3d-a50e-6397b29dc652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:56.317025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-72ac243b-3b49-4c3d-a50e-6397b29dc652 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:56.318238 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-72ac243b-3b49-4c3d-a50e-6397b29dc652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:56.375611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 988/1995] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:11:56.384000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30193d13-f9b2-4ef2-b21a-40d77acd83cb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:56.384287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30193d13-f9b2-4ef2-b21a-40d77acd83cb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:56.384560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30193d13-f9b2-4ef2-b21a-40d77acd83cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:56.405298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-30193d13-f9b2-4ef2-b21a-40d77acd83cb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:56.406465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-30193d13-f9b2-4ef2-b21a-40d77acd83cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:56.414788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1008/1996] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:56.419166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac243b-3b49-4c3d-a50e-6397b29dc652 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:56.429121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac243b-3b49-4c3d-a50e-6397b29dc652 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:56.434077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e4110-b305-4029-89b9-fce8d5dc6804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:56.434672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e4110-b305-4029-89b9-fce8d5dc6804 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:56.435025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e4110-b305-4029-89b9-fce8d5dc6804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:56.435344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8e4110-b305-4029-89b9-fce8d5dc6804 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:56.435980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e4110-b305-4029-89b9-fce8d5dc6804 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:11:56.436332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e4110-b305-4029-89b9-fce8d5dc6804 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:56.438012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8e4110-b305-4029-89b9-fce8d5dc6804 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:56.456348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 989/1997] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:11:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:11:56.461098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30193d13-f9b2-4ef2-b21a-40d77acd83cb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:56.472106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30193d13-f9b2-4ef2-b21a-40d77acd83cb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:56.477311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dece74-7af2-49c9-9138-b7aa6b9fc24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:56.477988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dece74-7af2-49c9-9138-b7aa6b9fc24c admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:56.478333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dece74-7af2-49c9-9138-b7aa6b9fc24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:56.478674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dece74-7af2-49c9-9138-b7aa6b9fc24c admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:56.479383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dece74-7af2-49c9-9138-b7aa6b9fc24c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:56.481426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dece74-7af2-49c9-9138-b7aa6b9fc24c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:56.513696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1009/1998] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:11:56 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:56.522516 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e4110-b305-4029-89b9-fce8d5dc6804 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:56.533209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8e4110-b305-4029-89b9-fce8d5dc6804 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:56.541070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:56.599245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c2a5b-950f-4ff0-a347-0a55d48355c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:56.599844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c2a5b-950f-4ff0-a347-0a55d48355c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:56.600119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c2a5b-950f-4ff0-a347-0a55d48355c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:56.600386 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c2a5b-950f-4ff0-a347-0a55d48355c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:56.601099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c2a5b-950f-4ff0-a347-0a55d48355c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:56.603048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c2a5b-950f-4ff0-a347-0a55d48355c0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:56.626388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 990/1999] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:11:56 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:11:56.630424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dece74-7af2-49c9-9138-b7aa6b9fc24c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:56.642199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dece74-7af2-49c9-9138-b7aa6b9fc24c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:56.646994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca540c5-aa48-4338-9fb9-7b200215d0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:56.647813 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca540c5-aa48-4338-9fb9-7b200215d0a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:56.648625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca540c5-aa48-4338-9fb9-7b200215d0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:56.648983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca540c5-aa48-4338-9fb9-7b200215d0a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:56.649750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca540c5-aa48-4338-9fb9-7b200215d0a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:56.651392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca540c5-aa48-4338-9fb9-7b200215d0a6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:56.663999 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1010/2000] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:56.668091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c2a5b-950f-4ff0-a347-0a55d48355c0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:56.678019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c2a5b-950f-4ff0-a347-0a55d48355c0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:56.683320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2fe65-c5bc-425c-b923-31cef5553718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:56.683984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2fe65-c5bc-425c-b923-31cef5553718 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:56.684261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2fe65-c5bc-425c-b923-31cef5553718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:56.684531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2fe65-c5bc-425c-b923-31cef5553718 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:56.685198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2fe65-c5bc-425c-b923-31cef5553718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:56.686891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f2fe65-c5bc-425c-b923-31cef5553718 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:56.698535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 991/2001] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:11:56.703066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca540c5-aa48-4338-9fb9-7b200215d0a6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:56.712981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca540c5-aa48-4338-9fb9-7b200215d0a6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:56.717607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c7b2f-7a7b-4591-8c95-db7f730dccf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:56.718250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c7b2f-7a7b-4591-8c95-db7f730dccf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:56.718530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c7b2f-7a7b-4591-8c95-db7f730dccf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:56.718803 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011c7b2f-7a7b-4591-8c95-db7f730dccf6 admin admin] PATH_INFO: `/v3/projects/fa35ab66591b4e3d8bb6b6551b6e38ea/users/2771232a37d64ad6b43afc18e380788e/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:56.733010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c7b2f-7a7b-4591-8c95-db7f730dccf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa35ab66591b4e3d8bb6b6551b6e38ea, user_id=2771232a37d64ad6b43afc18e380788e, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:56.735700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011c7b2f-7a7b-4591-8c95-db7f730dccf6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:56.748930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1011/2002] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:56 2022] PUT /identity/v3/projects/fa35ab66591b4e3d8bb6b6551b6e38ea/users/2771232a37d64ad6b43afc18e380788e/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:56.759793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a644d5-822a-4919-b166-bc7abcba73f6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:56.760085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a644d5-822a-4919-b166-bc7abcba73f6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:56.760364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a644d5-822a-4919-b166-bc7abcba73f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:56.791202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d7a644d5-822a-4919-b166-bc7abcba73f6 None None] MFA Rules not processed for user `2771232a37d64ad6b43afc18e380788e`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:56.792345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d7a644d5-822a-4919-b166-bc7abcba73f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:56.858055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 992/2003] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:56.868275 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c7b2f-7a7b-4591-8c95-db7f730dccf6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:56.933427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011c7b2f-7a7b-4591-8c95-db7f730dccf6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:56.938060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd11fcb-9889-4518-bfdd-32fccb0aeb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:56.938666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd11fcb-9889-4518-bfdd-32fccb0aeb92 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:56.938979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd11fcb-9889-4518-bfdd-32fccb0aeb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:56.939249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd11fcb-9889-4518-bfdd-32fccb0aeb92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:56.939885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd11fcb-9889-4518-bfdd-32fccb0aeb92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:56.941814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd11fcb-9889-4518-bfdd-32fccb0aeb92 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:56.973261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1012/2004] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:11:56 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:56.974358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a644d5-822a-4919-b166-bc7abcba73f6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:56.986531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a644d5-822a-4919-b166-bc7abcba73f6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:57.045020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4720a-b72e-4194-ae27-909652b738de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:57.045566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4720a-b72e-4194-ae27-909652b738de admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.045847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4720a-b72e-4194-ae27-909652b738de admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.046120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d4720a-b72e-4194-ae27-909652b738de admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.046765 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4720a-b72e-4194-ae27-909652b738de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:57.048725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d4720a-b72e-4194-ae27-909652b738de admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:57.073619 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 993/2005] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:11:56 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:57.077743 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd11fcb-9889-4518-bfdd-32fccb0aeb92 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:57.090838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd11fcb-9889-4518-bfdd-32fccb0aeb92 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:57.105019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e27b4-7e40-4a68-aa6e-e94fd9a75728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:57.105799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e27b4-7e40-4a68-aa6e-e94fd9a75728 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.106150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e27b4-7e40-4a68-aa6e-e94fd9a75728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.106512 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e27b4-7e40-4a68-aa6e-e94fd9a75728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.107227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e27b4-7e40-4a68-aa6e-e94fd9a75728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:57.108906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e27b4-7e40-4a68-aa6e-e94fd9a75728 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:57.120493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1013/2006] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:57.124676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4720a-b72e-4194-ae27-909652b738de admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:57.134982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d4720a-b72e-4194-ae27-909652b738de admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:57.139685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec233a0c-c246-4e07-b0ee-3dec835bbc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:57.140426 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec233a0c-c246-4e07-b0ee-3dec835bbc92 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.140702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec233a0c-c246-4e07-b0ee-3dec835bbc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.140970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec233a0c-c246-4e07-b0ee-3dec835bbc92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.141638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec233a0c-c246-4e07-b0ee-3dec835bbc92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:57.143236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec233a0c-c246-4e07-b0ee-3dec835bbc92 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:57.145508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:57.158024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 994/2007] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:57.163193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e27b4-7e40-4a68-aa6e-e94fd9a75728 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:57.169031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:57.176049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e27b4-7e40-4a68-aa6e-e94fd9a75728 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:57.182840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fdb08-5642-432a-b870-c8ee387e9f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:57.183505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fdb08-5642-432a-b870-c8ee387e9f1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.183788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fdb08-5642-432a-b870-c8ee387e9f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.184066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730fdb08-5642-432a-b870-c8ee387e9f1e admin admin] PATH_INFO: `/v3/projects/f1bf1b4b4b8c4bbd98fba370b3d54796/users/080796213d91405594594c8d0b881554/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.200308 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fdb08-5642-432a-b870-c8ee387e9f1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1bf1b4b4b8c4bbd98fba370b3d54796, user_id=080796213d91405594594c8d0b881554, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:57.203023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730fdb08-5642-432a-b870-c8ee387e9f1e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:57.216633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1014/2008] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:57 2022] PUT /identity/v3/projects/f1bf1b4b4b8c4bbd98fba370b3d54796/users/080796213d91405594594c8d0b881554/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:57.228902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088070e-4a2c-45f9-ad0d-7c46534075ea None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.229454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088070e-4a2c-45f9-ad0d-7c46534075ea None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.229938 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088070e-4a2c-45f9-ad0d-7c46534075ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.260073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5088070e-4a2c-45f9-ad0d-7c46534075ea None None] MFA Rules not processed for user `080796213d91405594594c8d0b881554`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:57.261430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5088070e-4a2c-45f9-ad0d-7c46534075ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:57.330835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 995/2009] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:57.335762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fdb08-5642-432a-b870-c8ee387e9f1e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:57.396081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730fdb08-5642-432a-b870-c8ee387e9f1e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:57.401279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22028d70-03bc-4923-904e-c0eb3b9c81f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:57.401871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22028d70-03bc-4923-904e-c0eb3b9c81f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.402144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22028d70-03bc-4923-904e-c0eb3b9c81f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.402418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22028d70-03bc-4923-904e-c0eb3b9c81f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.403035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22028d70-03bc-4923-904e-c0eb3b9c81f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:57.404906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22028d70-03bc-4923-904e-c0eb3b9c81f2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:57.431577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1015/2010] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:11:57 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:57.435696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088070e-4a2c-45f9-ad0d-7c46534075ea None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:57.446228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088070e-4a2c-45f9-ad0d-7c46534075ea None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:57.500503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c2a6b-a0bd-4961-8765-6fe2bd706930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:57.501597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c2a6b-a0bd-4961-8765-6fe2bd706930 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.501597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c2a6b-a0bd-4961-8765-6fe2bd706930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.501777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c2a6b-a0bd-4961-8765-6fe2bd706930 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.502258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c2a6b-a0bd-4961-8765-6fe2bd706930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:57.504159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c2a6b-a0bd-4961-8765-6fe2bd706930 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:57.526184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 996/2011] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:11:57 2022] POST /identity/v3/users => generated 296 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:11:57.530283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22028d70-03bc-4923-904e-c0eb3b9c81f2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:57.541700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22028d70-03bc-4923-904e-c0eb3b9c81f2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:57.546446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e4dc5-a297-4c81-ba0d-f88412935937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:57.547065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e4dc5-a297-4c81-ba0d-f88412935937 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.547369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e4dc5-a297-4c81-ba0d-f88412935937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.547707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e4dc5-a297-4c81-ba0d-f88412935937 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.548402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e4dc5-a297-4c81-ba0d-f88412935937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:57.550113 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02e4dc5-a297-4c81-ba0d-f88412935937 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:57.561917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1016/2012] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:57.566245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c2a6b-a0bd-4961-8765-6fe2bd706930 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:57.576269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c2a6b-a0bd-4961-8765-6fe2bd706930 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:57.581028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a3161-23df-4d00-be9f-3e34dab66e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:57.581625 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3161-23df-4d00-be9f-3e34dab66e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.581899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3161-23df-4d00-be9f-3e34dab66e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.582167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a3161-23df-4d00-be9f-3e34dab66e20 admin admin] PATH_INFO: `/v3/projects/8e724ab879eb49018f033259ba02fa4b/users/6454b3505645469392d537c6be82866c/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.597324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a3161-23df-4d00-be9f-3e34dab66e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e724ab879eb49018f033259ba02fa4b, user_id=6454b3505645469392d537c6be82866c, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:57.599940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a3161-23df-4d00-be9f-3e34dab66e20 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:57.613144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 997/2013] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:57 2022] PUT /identity/v3/projects/8e724ab879eb49018f033259ba02fa4b/users/6454b3505645469392d537c6be82866c/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:57.617089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e4dc5-a297-4c81-ba0d-f88412935937 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:57.626981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e4dc5-a297-4c81-ba0d-f88412935937 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:57.684752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda65a0-9a2f-4d8e-9bb1-094fc587e59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:57.685436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda65a0-9a2f-4d8e-9bb1-094fc587e59b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.685722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda65a0-9a2f-4d8e-9bb1-094fc587e59b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.686021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda65a0-9a2f-4d8e-9bb1-094fc587e59b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.686666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda65a0-9a2f-4d8e-9bb1-094fc587e59b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:57.688256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcda65a0-9a2f-4d8e-9bb1-094fc587e59b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:57.699495 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1017/2014] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:11:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:11:57.703565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a3161-23df-4d00-be9f-3e34dab66e20 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:57.714405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a3161-23df-4d00-be9f-3e34dab66e20 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:57.718944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e87f60-6c27-4631-b6c8-2cc6f60771b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:57.719588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e87f60-6c27-4631-b6c8-2cc6f60771b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.719865 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e87f60-6c27-4631-b6c8-2cc6f60771b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.720137 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e87f60-6c27-4631-b6c8-2cc6f60771b6 admin admin] PATH_INFO: `/v3/projects/8e724ab879eb49018f033259ba02fa4b/users/6454b3505645469392d537c6be82866c/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.722095 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e87f60-6c27-4631-b6c8-2cc6f60771b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e724ab879eb49018f033259ba02fa4b, user_id=6454b3505645469392d537c6be82866c, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:57.724740 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e87f60-6c27-4631-b6c8-2cc6f60771b6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:57.735411 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 998/2015] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:11:57 2022] PUT /identity/v3/projects/8e724ab879eb49018f033259ba02fa4b/users/6454b3505645469392d537c6be82866c/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:11:57.745440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404f11f-4bda-4961-bc91-b717063b3f67 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.745733 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404f11f-4bda-4961-bc91-b717063b3f67 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.746014 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a404f11f-4bda-4961-bc91-b717063b3f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.775020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a404f11f-4bda-4961-bc91-b717063b3f67 None None] MFA Rules not processed for user `6454b3505645469392d537c6be82866c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:11:57.776305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a404f11f-4bda-4961-bc91-b717063b3f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:57.815105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:57.849652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1018/2016] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:11:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:11:57.861322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e87f60-6c27-4631-b6c8-2cc6f60771b6 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:57.863078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-17e87f60-6c27-4631-b6c8-2cc6f60771b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:57.907150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:11:57.921410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e87f60-6c27-4631-b6c8-2cc6f60771b6 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:57.925081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370edced-3628-4efb-959b-5b5cf76a6db2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:57.925704 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370edced-3628-4efb-959b-5b5cf76a6db2 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:57.925984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370edced-3628-4efb-959b-5b5cf76a6db2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:57.926255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370edced-3628-4efb-959b-5b5cf76a6db2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:57.935315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370edced-3628-4efb-959b-5b5cf76a6db2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:57.936726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370edced-3628-4efb-959b-5b5cf76a6db2 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:57.946450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 999/2017] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:11:57.988345 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404f11f-4bda-4961-bc91-b717063b3f67 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:11:57.989797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a404f11f-4bda-4961-bc91-b717063b3f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:11:58.002103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a404f11f-4bda-4961-bc91-b717063b3f67 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:11:58.005784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc163e60-afab-49af-9100-2f4ad696917b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:11:58.006442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc163e60-afab-49af-9100-2f4ad696917b service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:11:58.006718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc163e60-afab-49af-9100-2f4ad696917b service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:11:58.007474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc163e60-afab-49af-9100-2f4ad696917b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:11:58.064138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc163e60-afab-49af-9100-2f4ad696917b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:11:58.065626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc163e60-afab-49af-9100-2f4ad696917b service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:11:58.075307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1019/2018] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:11:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:05.296070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370edced-3628-4efb-959b-5b5cf76a6db2 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:05.361358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370edced-3628-4efb-959b-5b5cf76a6db2 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:05.366207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:05.367373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:05.367650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:05.367921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] PATH_INFO: `/v3/users/2771232a37d64ad6b43afc18e380788e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:05.369082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2771232a37d64ad6b43afc18e380788e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:05.371042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:05.421311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2771232a37d64ad6b43afc18e380788e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:05.422037 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2771232a37d64ad6b43afc18e380788e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:05.430382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2771232a37d64ad6b43afc18e380788e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:05.431054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2771232a37d64ad6b43afc18e380788e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:05.443400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2771232a37d64ad6b43afc18e380788e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:05.456277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1000/2019] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:05 2022] DELETE /identity/v3/users/2771232a37d64ad6b43afc18e380788e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:05.461438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2120-3286-41b1-aaa7-95d97f338e39 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:05.461910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2120-3286-41b1-aaa7-95d97f338e39 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:05.462077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad2120-3286-41b1-aaa7-95d97f338e39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:05.482044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7fad2120-3286-41b1-aaa7-95d97f338e39 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:05.483229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7fad2120-3286-41b1-aaa7-95d97f338e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:05.545245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1020/2020] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:05.553332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:05.555005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:05.614094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35f683c-2577-46b2-8b72-2ad23b78e544 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:05.618102 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa310c-320a-427b-9520-474e26352324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:05.618712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa310c-320a-427b-9520-474e26352324 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:05.618988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa310c-320a-427b-9520-474e26352324 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:05.619259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa310c-320a-427b-9520-474e26352324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:05.628963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa310c-320a-427b-9520-474e26352324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:05.630540 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa310c-320a-427b-9520-474e26352324 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:05.640721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1001/2021] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:05.921474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad2120-3286-41b1-aaa7-95d97f338e39 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:05.923010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7fad2120-3286-41b1-aaa7-95d97f338e39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:05.950759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad2120-3286-41b1-aaa7-95d97f338e39 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:05.956374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792402ae-eaa0-4738-8092-0ac8616e8a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:05.956838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792402ae-eaa0-4738-8092-0ac8616e8a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:05.957224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792402ae-eaa0-4738-8092-0ac8616e8a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:05.957573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792402ae-eaa0-4738-8092-0ac8616e8a0e admin admin] PATH_INFO: `/v3/users/747337ac2f6f46f3b26f18c4c8ec22bd` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:05.958863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792402ae-eaa0-4738-8092-0ac8616e8a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=747337ac2f6f46f3b26f18c4c8ec22bd)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:05.960852 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792402ae-eaa0-4738-8092-0ac8616e8a0e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:06.044489 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-792402ae-eaa0-4738-8092-0ac8616e8a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '747337ac2f6f46f3b26f18c4c8ec22bd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.045279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-792402ae-eaa0-4738-8092-0ac8616e8a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '747337ac2f6f46f3b26f18c4c8ec22bd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.046246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-792402ae-eaa0-4738-8092-0ac8616e8a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '747337ac2f6f46f3b26f18c4c8ec22bd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.083066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-792402ae-eaa0-4738-8092-0ac8616e8a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '747337ac2f6f46f3b26f18c4c8ec22bd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.086575 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:06.090626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa310c-320a-427b-9520-474e26352324 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:06.092651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-05fa310c-320a-427b-9520-474e26352324 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:06.096837 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-792402ae-eaa0-4738-8092-0ac8616e8a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '747337ac2f6f46f3b26f18c4c8ec22bd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.115442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:06.116971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1021/2022] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:05 2022] DELETE /identity/v3/users/747337ac2f6f46f3b26f18c4c8ec22bd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:06.122212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e07a8-9a3f-4c81-8edf-8c5031d078c3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:06.122587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e07a8-9a3f-4c81-8edf-8c5031d078c3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:06.123082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e07a8-9a3f-4c81-8edf-8c5031d078c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:06.128755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa310c-320a-427b-9520-474e26352324 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:06.155601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c52e07a8-9a3f-4c81-8edf-8c5031d078c3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:06.157496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c52e07a8-9a3f-4c81-8edf-8c5031d078c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:06.218044 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:06.219947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:06.220237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:06.220509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] PATH_INFO: `/v3/projects/fa35ab66591b4e3d8bb6b6551b6e38ea` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:06.221853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa35ab66591b4e3d8bb6b6551b6e38ea)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:06.223993 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:06.242492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1022/2023] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:06.249459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e07a8-9a3f-4c81-8edf-8c5031d078c3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:06.250879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c52e07a8-9a3f-4c81-8edf-8c5031d078c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:06.289818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa35ab66591b4e3d8bb6b6551b6e38ea', 'request_id': 'req-39205229-d604-44cd-b533-535db10cefe8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.299121 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa35ab66591b4e3d8bb6b6551b6e38ea', 'request_id': 'req-39205229-d604-44cd-b533-535db10cefe8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.308324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa35ab66591b4e3d8bb6b6551b6e38ea', 'request_id': 'req-39205229-d604-44cd-b533-535db10cefe8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.314768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e07a8-9a3f-4c81-8edf-8c5031d078c3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:06.321063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] The token cache is being invalidate because project fa35ab66591b4e3d8bb6b6551b6e38ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:06.321389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.322045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.323542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1002/2024] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:06 2022] DELETE /identity/v3/projects/fa35ab66591b4e3d8bb6b6551b6e38ea => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:06.327414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:06.328730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:06.365781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fda33a-2a55-4208-9db8-b72a0f91e236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:06.366415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fda33a-2a55-4208-9db8-b72a0f91e236 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:06.366676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fda33a-2a55-4208-9db8-b72a0f91e236 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:06.366940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fda33a-2a55-4208-9db8-b72a0f91e236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:06.368727 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-03fda33a-2a55-4208-9db8-b72a0f91e236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:06.394774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39205229-d604-44cd-b533-535db10cefe8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:06.400257 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:06.400909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:06.401223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:06.401502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] PATH_INFO: `/v3/users/080796213d91405594594c8d0b881554` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:06.402646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=080796213d91405594594c8d0b881554)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:06.404626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:06.422007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:06.437117 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fda33a-2a55-4208-9db8-b72a0f91e236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:06.438863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fda33a-2a55-4208-9db8-b72a0f91e236 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:06.448164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:06.450080 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1023/2025] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:12:06.458409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080796213d91405594594c8d0b881554'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.459130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080796213d91405594594c8d0b881554'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.467406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080796213d91405594594c8d0b881554'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.468103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080796213d91405594594c8d0b881554'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.482998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080796213d91405594594c8d0b881554'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:06.498598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1003/2026] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:06 2022] DELETE /identity/v3/users/080796213d91405594594c8d0b881554 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:06.903843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fda33a-2a55-4208-9db8-b72a0f91e236 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:06.906247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-03fda33a-2a55-4208-9db8-b72a0f91e236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:06.910557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:06.912137 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:06.926821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fda33a-2a55-4208-9db8-b72a0f91e236 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:07.008619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:07.009304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:07.009583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:07.009921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] PATH_INFO: `/v3/projects/f71a3207ec8f464792e63f7d78e5c946` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:07.011126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f71a3207ec8f464792e63f7d78e5c946)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:07.014029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:07.031295 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47898b9e-926a-4681-a05e-0c1069127933 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:07.036779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:07.038211 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:07.038630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:07.039042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] PATH_INFO: `/v3/projects/f1bf1b4b4b8c4bbd98fba370b3d54796` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:07.040667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1bf1b4b4b8c4bbd98fba370b3d54796)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:07.043371 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:07.090495 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f71a3207ec8f464792e63f7d78e5c946', 'request_id': 'req-9cba639d-9839-4d5c-8698-ca0f231e825f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.098161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f71a3207ec8f464792e63f7d78e5c946', 'request_id': 'req-9cba639d-9839-4d5c-8698-ca0f231e825f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.107782 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f71a3207ec8f464792e63f7d78e5c946', 'request_id': 'req-9cba639d-9839-4d5c-8698-ca0f231e825f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.115997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1bf1b4b4b8c4bbd98fba370b3d54796', 'request_id': 'req-3ccd9507-2d85-4bcf-bab2-5e222764feab'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.124625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] The token cache is being invalidate because project f71a3207ec8f464792e63f7d78e5c946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:07.124948 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.125633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.125778 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1bf1b4b4b8c4bbd98fba370b3d54796', 'request_id': 'req-3ccd9507-2d85-4bcf-bab2-5e222764feab'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.127692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1024/2027] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:06 2022] DELETE /identity/v3/projects/f71a3207ec8f464792e63f7d78e5c946 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:07.131485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:07.133194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:07.135702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1bf1b4b4b8c4bbd98fba370b3d54796', 'request_id': 'req-3ccd9507-2d85-4bcf-bab2-5e222764feab'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.161434 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] The token cache is being invalidate because project f1bf1b4b4b8c4bbd98fba370b3d54796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:07.161434 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.161434 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.161434 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1004/2028] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:06 2022] DELETE /identity/v3/projects/f1bf1b4b4b8c4bbd98fba370b3d54796 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:07.168853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:07.170299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:07.219515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba639d-9839-4d5c-8698-ca0f231e825f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:07.229265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:07.251018 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd9507-2d85-4bcf-bab2-5e222764feab admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:07.254966 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:07.255778 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:07.256783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:07.257294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:07.257689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] PATH_INFO: `/v3/users/6454b3505645469392d537c6be82866c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:07.258898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6454b3505645469392d537c6be82866c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:07.261267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:07.290227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:07.290874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:07.291220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:07.291494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] PATH_INFO: `/v3/users/8704bba52e754068a17957ac57800dd6` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:07.292716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8704bba52e754068a17957ac57800dd6)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:07.294865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:07.311007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6454b3505645469392d537c6be82866c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.311673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6454b3505645469392d537c6be82866c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.319477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6454b3505645469392d537c6be82866c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.320936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6454b3505645469392d537c6be82866c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.340401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6454b3505645469392d537c6be82866c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.349165 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8704bba52e754068a17957ac57800dd6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.349875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8704bba52e754068a17957ac57800dd6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.351138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8704bba52e754068a17957ac57800dd6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.354438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1005/2029] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:07 2022] DELETE /identity/v3/users/6454b3505645469392d537c6be82866c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:07.363378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8704bba52e754068a17957ac57800dd6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.370572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8704bba52e754068a17957ac57800dd6'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.383524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1025/2030] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:07 2022] DELETE /identity/v3/users/8704bba52e754068a17957ac57800dd6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:07.651822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:07.653950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:07.659809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:07.665805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:07.739030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b610e2-90fe-499e-b3c5-38fe16fae48d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:07.745503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae883273-7928-451f-aab1-53dd671f0071 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:07.745940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:07.746580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:07.746861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:07.747141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] PATH_INFO: `/v3/projects/25941b21709344cfbfe794b6d563906d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:07.748601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25941b21709344cfbfe794b6d563906d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:07.750376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:07.750669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:07.751007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:07.751291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:07.751564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] PATH_INFO: `/v3/projects/8e724ab879eb49018f033259ba02fa4b` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:07.752693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e724ab879eb49018f033259ba02fa4b)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:07.756872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:07.813265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e724ab879eb49018f033259ba02fa4b', 'request_id': 'req-8e960864-83b5-4c50-9326-8d5820cf4321'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.814609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25941b21709344cfbfe794b6d563906d', 'request_id': 'req-d0142e63-8722-4cf6-8f60-c790adff4e85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.815770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:07.824203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e724ab879eb49018f033259ba02fa4b', 'request_id': 'req-8e960864-83b5-4c50-9326-8d5820cf4321'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.826834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25941b21709344cfbfe794b6d563906d', 'request_id': 'req-d0142e63-8722-4cf6-8f60-c790adff4e85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.831809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e724ab879eb49018f033259ba02fa4b', 'request_id': 'req-8e960864-83b5-4c50-9326-8d5820cf4321'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.833667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25941b21709344cfbfe794b6d563906d', 'request_id': 'req-d0142e63-8722-4cf6-8f60-c790adff4e85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.837042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:07.848202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] The token cache is being invalidate because project 8e724ab879eb49018f033259ba02fa4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:07.848512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.849935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8e960864-83b5-4c50-9326-8d5820cf4321 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.852989 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1006/2031] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:07 2022] DELETE /identity/v3/projects/8e724ab879eb49018f033259ba02fa4b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:07.853185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] The token cache is being invalidate because project 25941b21709344cfbfe794b6d563906d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:07.853502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.854250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d0142e63-8722-4cf6-8f60-c790adff4e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:07.855889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1026/2032] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:07 2022] DELETE /identity/v3/projects/25941b21709344cfbfe794b6d563906d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:07.858977 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd93e62-17d0-4772-8fb1-6fa7d94f2769 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:07.859765 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd93e62-17d0-4772-8fb1-6fa7d94f2769 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:07.860079 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd93e62-17d0-4772-8fb1-6fa7d94f2769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:07.860196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680049f2-969b-4db7-a881-cad906045ffc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:07.860491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680049f2-969b-4db7-a881-cad906045ffc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:07.860773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680049f2-969b-4db7-a881-cad906045ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:07.882526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6cd93e62-17d0-4772-8fb1-6fa7d94f2769 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:07.884172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6cd93e62-17d0-4772-8fb1-6fa7d94f2769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:07.888939 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-680049f2-969b-4db7-a881-cad906045ffc None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:07.890191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-680049f2-969b-4db7-a881-cad906045ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:07.951949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1027/2033] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:07.954405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1007/2034] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:07.961461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e7657-7143-4109-9b26-8280c65c1c8b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:07.961739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e7657-7143-4109-9b26-8280c65c1c8b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:07.962007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e7657-7143-4109-9b26-8280c65c1c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:07.964436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10597e02-0e8f-4da8-b7e8-e365aeb9f457 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:07.964723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10597e02-0e8f-4da8-b7e8-e365aeb9f457 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:07.965000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10597e02-0e8f-4da8-b7e8-e365aeb9f457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:07.984113 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d73e7657-7143-4109-9b26-8280c65c1c8b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:07.985369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d73e7657-7143-4109-9b26-8280c65c1c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:07.985628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-10597e02-0e8f-4da8-b7e8-e365aeb9f457 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:07.986851 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-10597e02-0e8f-4da8-b7e8-e365aeb9f457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:07.993371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1028/2035] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:07.995714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1008/2036] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:07.999085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e7657-7143-4109-9b26-8280c65c1c8b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:07.999829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10597e02-0e8f-4da8-b7e8-e365aeb9f457 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.010382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e7657-7143-4109-9b26-8280c65c1c8b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.010619 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10597e02-0e8f-4da8-b7e8-e365aeb9f457 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.014623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefbfe2-7e1f-4134-8a60-74efaf64f5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.015221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefbfe2-7e1f-4134-8a60-74efaf64f5ca admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.015366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20346402-c331-4b70-b0b6-b7d1fb3a2081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.015485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefbfe2-7e1f-4134-8a60-74efaf64f5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.015756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefbfe2-7e1f-4134-8a60-74efaf64f5ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.015873 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20346402-c331-4b70-b0b6-b7d1fb3a2081 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.016119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20346402-c331-4b70-b0b6-b7d1fb3a2081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.016388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20346402-c331-4b70-b0b6-b7d1fb3a2081 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.016495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefbfe2-7e1f-4134-8a60-74efaf64f5ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:12:08.016687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefbfe2-7e1f-4134-8a60-74efaf64f5ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.017027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20346402-c331-4b70-b0b6-b7d1fb3a2081 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:12:08.017348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20346402-c331-4b70-b0b6-b7d1fb3a2081 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.018330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefbfe2-7e1f-4134-8a60-74efaf64f5ca admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.018922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20346402-c331-4b70-b0b6-b7d1fb3a2081 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.031416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1009/2037] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:12:08.036024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefbfe2-7e1f-4134-8a60-74efaf64f5ca admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.037271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1029/2038] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:12:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:12:08.042137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20346402-c331-4b70-b0b6-b7d1fb3a2081 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.049363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefbfe2-7e1f-4134-8a60-74efaf64f5ca admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.052567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20346402-c331-4b70-b0b6-b7d1fb3a2081 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.060910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab3966-e186-4c98-9a48-275bc90b5a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.061892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab3966-e186-4c98-9a48-275bc90b5a2b admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.062026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab3966-e186-4c98-9a48-275bc90b5a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.062168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6383ec-a824-421c-b9cf-d976086cd3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.062381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dab3966-e186-4c98-9a48-275bc90b5a2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.062926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6383ec-a824-421c-b9cf-d976086cd3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.063024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab3966-e186-4c98-9a48-275bc90b5a2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.063272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6383ec-a824-421c-b9cf-d976086cd3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.063620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6383ec-a824-421c-b9cf-d976086cd3e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.064314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6383ec-a824-421c-b9cf-d976086cd3e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.064979 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dab3966-e186-4c98-9a48-275bc90b5a2b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.066340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6383ec-a824-421c-b9cf-d976086cd3e9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.101180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1010/2039] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:08 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:08.105023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab3966-e186-4c98-9a48-275bc90b5a2b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.107610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1030/2040] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:08 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:08.108609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6383ec-a824-421c-b9cf-d976086cd3e9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.179488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dab3966-e186-4c98-9a48-275bc90b5a2b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.205546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6383ec-a824-421c-b9cf-d976086cd3e9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.212877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f14d1-5dde-4fab-ab00-b6e876feee7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.214422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f14d1-5dde-4fab-ab00-b6e876feee7e admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.214988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f14d1-5dde-4fab-ab00-b6e876feee7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.215465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f14d1-5dde-4fab-ab00-b6e876feee7e admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.216419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f14d1-5dde-4fab-ab00-b6e876feee7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.219146 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f14d1-5dde-4fab-ab00-b6e876feee7e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.252736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1031/2041] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:08 2022] POST /identity/v3/users => generated 283 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:08.260419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f14d1-5dde-4fab-ab00-b6e876feee7e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.271293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee618a-a068-4497-8fa3-c5d4b18c202b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.272590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee618a-a068-4497-8fa3-c5d4b18c202b admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.272964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee618a-a068-4497-8fa3-c5d4b18c202b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.273302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee618a-a068-4497-8fa3-c5d4b18c202b admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.273911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee618a-a068-4497-8fa3-c5d4b18c202b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.275953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee618a-a068-4497-8fa3-c5d4b18c202b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.285810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:08.322906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1011/2042] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:08 2022] POST /identity/v3/users => generated 284 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:08.326204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee618a-a068-4497-8fa3-c5d4b18c202b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.337787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f14d1-5dde-4fab-ab00-b6e876feee7e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.347075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee618a-a068-4497-8fa3-c5d4b18c202b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.347908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93811095-d149-4e0b-927e-53e7543fd133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.349898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93811095-d149-4e0b-927e-53e7543fd133 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.350262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93811095-d149-4e0b-927e-53e7543fd133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.350507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93811095-d149-4e0b-927e-53e7543fd133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.351268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93811095-d149-4e0b-927e-53e7543fd133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.353273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93811095-d149-4e0b-927e-53e7543fd133 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.358265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249277ba-de49-4aef-92b9-bf54b3903258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.359324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249277ba-de49-4aef-92b9-bf54b3903258 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.359849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249277ba-de49-4aef-92b9-bf54b3903258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.360344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249277ba-de49-4aef-92b9-bf54b3903258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.361354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249277ba-de49-4aef-92b9-bf54b3903258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.363427 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249277ba-de49-4aef-92b9-bf54b3903258 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.379907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1032/2043] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:08.387006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93811095-d149-4e0b-927e-53e7543fd133 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.388776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1012/2044] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:08.392827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249277ba-de49-4aef-92b9-bf54b3903258 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.411879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93811095-d149-4e0b-927e-53e7543fd133 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.413921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249277ba-de49-4aef-92b9-bf54b3903258 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.418023 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e13eb-378a-4f7b-a9be-29ccc16052d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.419099 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e13eb-378a-4f7b-a9be-29ccc16052d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.419236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755c05-f8e5-4ff9-bd87-c0aed2657028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.419394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e13eb-378a-4f7b-a9be-29ccc16052d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.419656 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e13eb-378a-4f7b-a9be-29ccc16052d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.420529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755c05-f8e5-4ff9-bd87-c0aed2657028 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.420642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e13eb-378a-4f7b-a9be-29ccc16052d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.420718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755c05-f8e5-4ff9-bd87-c0aed2657028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.420718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f755c05-f8e5-4ff9-bd87-c0aed2657028 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.421489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f755c05-f8e5-4ff9-bd87-c0aed2657028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.422289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e13eb-378a-4f7b-a9be-29ccc16052d1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.423258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f755c05-f8e5-4ff9-bd87-c0aed2657028 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.434279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1033/2045] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:08.439176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e13eb-378a-4f7b-a9be-29ccc16052d1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.439810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1013/2046] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:12:08.445613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755c05-f8e5-4ff9-bd87-c0aed2657028 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.446696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:08.451492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e13eb-378a-4f7b-a9be-29ccc16052d1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.458500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f755c05-f8e5-4ff9-bd87-c0aed2657028 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.460384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3695f-a4c6-495a-b135-a16454a56590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.461575 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3695f-a4c6-495a-b135-a16454a56590 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.462325 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3695f-a4c6-495a-b135-a16454a56590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.463182 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3695f-a4c6-495a-b135-a16454a56590 admin admin] PATH_INFO: `/v3/projects/44a035550b2d402cbb7f87a031a6d5b4/users/334bb3ffcafb40348522a25f1f8357dc/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.464360 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ae9c51-ee20-422c-9082-b50f9fc1f7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.465344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae9c51-ee20-422c-9082-b50f9fc1f7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.465624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae9c51-ee20-422c-9082-b50f9fc1f7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.466680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae9c51-ee20-422c-9082-b50f9fc1f7d6 admin admin] PATH_INFO: `/v3/projects/a3ec7e60c54d4fa19f6e5e839e99e220/users/689e9243133b4401979a4a6d2f30c22d/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.478942 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3695f-a4c6-495a-b135-a16454a56590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44a035550b2d402cbb7f87a031a6d5b4, user_id=334bb3ffcafb40348522a25f1f8357dc, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.482679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3695f-a4c6-495a-b135-a16454a56590 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.483784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae9c51-ee20-422c-9082-b50f9fc1f7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3ec7e60c54d4fa19f6e5e839e99e220, user_id=689e9243133b4401979a4a6d2f30c22d, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.487041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ae9c51-ee20-422c-9082-b50f9fc1f7d6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.499647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1034/2047] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:08 2022] PUT /identity/v3/projects/44a035550b2d402cbb7f87a031a6d5b4/users/334bb3ffcafb40348522a25f1f8357dc/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:08.509351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1014/2048] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:08 2022] PUT /identity/v3/projects/a3ec7e60c54d4fa19f6e5e839e99e220/users/689e9243133b4401979a4a6d2f30c22d/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:08.510935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81814db0-ba3c-4c91-a5e8-9a9ea8f86f0a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.511282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81814db0-ba3c-4c91-a5e8-9a9ea8f86f0a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.511611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81814db0-ba3c-4c91-a5e8-9a9ea8f86f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.520508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2078ec-d68f-4f7a-8c51-83508bd47279 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.520796 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2078ec-d68f-4f7a-8c51-83508bd47279 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.521074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2078ec-d68f-4f7a-8c51-83508bd47279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.549092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-81814db0-ba3c-4c91-a5e8-9a9ea8f86f0a None None] MFA Rules not processed for user `334bb3ffcafb40348522a25f1f8357dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:08.550526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-81814db0-ba3c-4c91-a5e8-9a9ea8f86f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:08.556918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fd2078ec-d68f-4f7a-8c51-83508bd47279 None None] MFA Rules not processed for user `689e9243133b4401979a4a6d2f30c22d`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:08.558182 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fd2078ec-d68f-4f7a-8c51-83508bd47279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:08.624336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1035/2049] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:08.625214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1015/2050] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:08.629634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81814db0-ba3c-4c91-a5e8-9a9ea8f86f0a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.632320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2078ec-d68f-4f7a-8c51-83508bd47279 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.695365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2078ec-d68f-4f7a-8c51-83508bd47279 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.696260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81814db0-ba3c-4c91-a5e8-9a9ea8f86f0a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.700108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8b18d-06e3-496c-aee3-47a75f6a4f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.700718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8b18d-06e3-496c-aee3-47a75f6a4f5d admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.700885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ebb5b-558a-4eb9-b714-eeec05e84888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.701010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8b18d-06e3-496c-aee3-47a75f6a4f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.701296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8b18d-06e3-496c-aee3-47a75f6a4f5d admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.701550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ebb5b-558a-4eb9-b714-eeec05e84888 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.701831 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ebb5b-558a-4eb9-b714-eeec05e84888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.701923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8b18d-06e3-496c-aee3-47a75f6a4f5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.702103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ebb5b-558a-4eb9-b714-eeec05e84888 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.702745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ebb5b-558a-4eb9-b714-eeec05e84888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.703823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8b18d-06e3-496c-aee3-47a75f6a4f5d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.704623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ebb5b-558a-4eb9-b714-eeec05e84888 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.733294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1036/2051] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:08 2022] POST /identity/v3/projects => generated 365 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:08.735517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1016/2052] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:08 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:08.737488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ebb5b-558a-4eb9-b714-eeec05e84888 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.740049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8b18d-06e3-496c-aee3-47a75f6a4f5d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.759168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:08.799844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:08.804798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8b18d-06e3-496c-aee3-47a75f6a4f5d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.808487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ebb5b-558a-4eb9-b714-eeec05e84888 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.809550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbd635-99ae-4fa3-8a22-50b7df8fb4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.810156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbd635-99ae-4fa3-8a22-50b7df8fb4e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.810433 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbd635-99ae-4fa3-8a22-50b7df8fb4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.810731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dbd635-99ae-4fa3-8a22-50b7df8fb4e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.811366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dbd635-99ae-4fa3-8a22-50b7df8fb4e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.813324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dbd635-99ae-4fa3-8a22-50b7df8fb4e9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.813426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deef1da-8b00-475d-84b5-0965097256f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.814027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deef1da-8b00-475d-84b5-0965097256f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.814302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deef1da-8b00-475d-84b5-0965097256f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.814576 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deef1da-8b00-475d-84b5-0965097256f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.815222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deef1da-8b00-475d-84b5-0965097256f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.817181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deef1da-8b00-475d-84b5-0965097256f4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.835190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1017/2053] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:08 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:08.838463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1037/2054] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:08 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:08.839373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbd635-99ae-4fa3-8a22-50b7df8fb4e9 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.842003 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deef1da-8b00-475d-84b5-0965097256f4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.849135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dbd635-99ae-4fa3-8a22-50b7df8fb4e9 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.852474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deef1da-8b00-475d-84b5-0965097256f4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.854488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539b3d8-a3d7-4424-a5c5-b766a42fad37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.855077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b3d8-a3d7-4424-a5c5-b766a42fad37 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.855352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b3d8-a3d7-4424-a5c5-b766a42fad37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.855619 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539b3d8-a3d7-4424-a5c5-b766a42fad37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.856246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539b3d8-a3d7-4424-a5c5-b766a42fad37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.857001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad21f77-9266-4692-996d-7f14da0d3745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.857621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad21f77-9266-4692-996d-7f14da0d3745 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.857895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad21f77-9266-4692-996d-7f14da0d3745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.858162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad21f77-9266-4692-996d-7f14da0d3745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.858834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad21f77-9266-4692-996d-7f14da0d3745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.859039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a539b3d8-a3d7-4424-a5c5-b766a42fad37 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.860348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad21f77-9266-4692-996d-7f14da0d3745 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.870070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1018/2055] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:12:08.873291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1038/2056] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:08.874377 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539b3d8-a3d7-4424-a5c5-b766a42fad37 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.874509 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad21f77-9266-4692-996d-7f14da0d3745 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.884083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539b3d8-a3d7-4424-a5c5-b766a42fad37 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.885693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad21f77-9266-4692-996d-7f14da0d3745 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:08.888974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecdf66-f34c-441a-acd9-73c98558c26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.889636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecdf66-f34c-441a-acd9-73c98558c26e admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.889913 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecdf66-f34c-441a-acd9-73c98558c26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.890180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecdf66-f34c-441a-acd9-73c98558c26e admin admin] PATH_INFO: `/v3/projects/b12b9023e05f4591bc945451b9457059/users/50d668ac07a142bd846df7c521943b1d/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.890382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666569d-36eb-4ac7-b0f4-ae1d49f95123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:08.891025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666569d-36eb-4ac7-b0f4-ae1d49f95123 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:08.891305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666569d-36eb-4ac7-b0f4-ae1d49f95123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:08.891582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9666569d-36eb-4ac7-b0f4-ae1d49f95123 admin admin] PATH_INFO: `/v3/projects/29810d29944a44ef8e9d4c4dba7e49cc/users/ce9210ed36d4453bb5d7a79eae236ca8/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:08.906437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ecdf66-f34c-441a-acd9-73c98558c26e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b12b9023e05f4591bc945451b9457059, user_id=50d668ac07a142bd846df7c521943b1d, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.907451 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666569d-36eb-4ac7-b0f4-ae1d49f95123 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29810d29944a44ef8e9d4c4dba7e49cc, user_id=ce9210ed36d4453bb5d7a79eae236ca8, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:08.909150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ecdf66-f34c-441a-acd9-73c98558c26e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.910089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9666569d-36eb-4ac7-b0f4-ae1d49f95123 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:08.922424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1019/2057] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:08 2022] PUT /identity/v3/projects/b12b9023e05f4591bc945451b9457059/users/50d668ac07a142bd846df7c521943b1d/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:08.923234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1039/2058] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:08 2022] PUT /identity/v3/projects/29810d29944a44ef8e9d4c4dba7e49cc/users/ce9210ed36d4453bb5d7a79eae236ca8/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:08.926019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecdf66-f34c-441a-acd9-73c98558c26e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.927047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666569d-36eb-4ac7-b0f4-ae1d49f95123 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:08.996839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9666569d-36eb-4ac7-b0f4-ae1d49f95123 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:09.002300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2e6a9-104c-48f5-95cd-590dd6d6c4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:09.002977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2e6a9-104c-48f5-95cd-590dd6d6c4bb admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:09.003323 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2e6a9-104c-48f5-95cd-590dd6d6c4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:09.003665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2e6a9-104c-48f5-95cd-590dd6d6c4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:09.003981 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ecdf66-f34c-441a-acd9-73c98558c26e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:09.004442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2e6a9-104c-48f5-95cd-590dd6d6c4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:09.006121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2e6a9-104c-48f5-95cd-590dd6d6c4bb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:09.009972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e075dc-0923-417d-999a-68c4710b2a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:09.010886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e075dc-0923-417d-999a-68c4710b2a42 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:09.011300 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e075dc-0923-417d-999a-68c4710b2a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:09.011710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e075dc-0923-417d-999a-68c4710b2a42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:09.013268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e075dc-0923-417d-999a-68c4710b2a42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:09.016036 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e075dc-0923-417d-999a-68c4710b2a42 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:09.017631 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1040/2059] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:09.022077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2e6a9-104c-48f5-95cd-590dd6d6c4bb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:09.032908 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1020/2060] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:09.035978 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2e6a9-104c-48f5-95cd-590dd6d6c4bb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:09.036515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e075dc-0923-417d-999a-68c4710b2a42 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:09.041597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f04fa3-999d-4ac9-9743-1312cbbe9b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:09.042257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f04fa3-999d-4ac9-9743-1312cbbe9b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:09.042529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f04fa3-999d-4ac9-9743-1312cbbe9b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:09.042801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f04fa3-999d-4ac9-9743-1312cbbe9b05 admin admin] PATH_INFO: `/v3/projects/29810d29944a44ef8e9d4c4dba7e49cc/users/ce9210ed36d4453bb5d7a79eae236ca8/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:09.044806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f04fa3-999d-4ac9-9743-1312cbbe9b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29810d29944a44ef8e9d4c4dba7e49cc, user_id=ce9210ed36d4453bb5d7a79eae236ca8, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:09.047347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e075dc-0923-417d-999a-68c4710b2a42 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:09.047473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f04fa3-999d-4ac9-9743-1312cbbe9b05 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:09.052195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e328b8e9-32e3-4223-b366-12e4451b3243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:09.053156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328b8e9-32e3-4223-b366-12e4451b3243 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:09.054036 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328b8e9-32e3-4223-b366-12e4451b3243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:09.054306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e328b8e9-32e3-4223-b366-12e4451b3243 admin admin] PATH_INFO: `/v3/projects/b12b9023e05f4591bc945451b9457059/users/50d668ac07a142bd846df7c521943b1d/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:09.056464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328b8e9-32e3-4223-b366-12e4451b3243 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b12b9023e05f4591bc945451b9457059, user_id=50d668ac07a142bd846df7c521943b1d, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:09.059574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e328b8e9-32e3-4223-b366-12e4451b3243 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:09.071457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1041/2061] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:09 2022] PUT /identity/v3/projects/29810d29944a44ef8e9d4c4dba7e49cc/users/ce9210ed36d4453bb5d7a79eae236ca8/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:09.075726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1021/2062] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:09 2022] PUT /identity/v3/projects/b12b9023e05f4591bc945451b9457059/users/50d668ac07a142bd846df7c521943b1d/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:09.081779 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5427d-c187-4d13-af39-0da1d781b02c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:09.082073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5427d-c187-4d13-af39-0da1d781b02c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:09.082353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5427d-c187-4d13-af39-0da1d781b02c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:09.085947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89999ca5-ed79-41d2-8510-0f3daba4f526 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:09.086309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89999ca5-ed79-41d2-8510-0f3daba4f526 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:09.086593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89999ca5-ed79-41d2-8510-0f3daba4f526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:09.114553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f3c5427d-c187-4d13-af39-0da1d781b02c None None] MFA Rules not processed for user `ce9210ed36d4453bb5d7a79eae236ca8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:09.116362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f3c5427d-c187-4d13-af39-0da1d781b02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:09.119288 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-89999ca5-ed79-41d2-8510-0f3daba4f526 None None] MFA Rules not processed for user `50d668ac07a142bd846df7c521943b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:09.120516 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-89999ca5-ed79-41d2-8510-0f3daba4f526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:09.166029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:09.189616 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1042/2063] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:09.192334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1022/2064] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:09 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:09.201518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5427d-c187-4d13-af39-0da1d781b02c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:09.202946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f3c5427d-c187-4d13-af39-0da1d781b02c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:09.203718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89999ca5-ed79-41d2-8510-0f3daba4f526 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:09.205096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-89999ca5-ed79-41d2-8510-0f3daba4f526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:09.223022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:09.262261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5427d-c187-4d13-af39-0da1d781b02c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:09.263413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89999ca5-ed79-41d2-8510-0f3daba4f526 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:09.265932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d783e76-27c2-413c-9b98-6882cfbca30b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:09.266537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d783e76-27c2-413c-9b98-6882cfbca30b service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:09.266814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d783e76-27c2-413c-9b98-6882cfbca30b service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:09.267084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d783e76-27c2-413c-9b98-6882cfbca30b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:09.267259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d530907-5a50-4bb0-809c-25ebc53ac68f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:09.267856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d530907-5a50-4bb0-809c-25ebc53ac68f service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:09.268149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d530907-5a50-4bb0-809c-25ebc53ac68f service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:09.268420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d530907-5a50-4bb0-809c-25ebc53ac68f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:09.276174 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d783e76-27c2-413c-9b98-6882cfbca30b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:09.277645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d783e76-27c2-413c-9b98-6882cfbca30b service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:09.278105 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d530907-5a50-4bb0-809c-25ebc53ac68f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:09.279600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d530907-5a50-4bb0-809c-25ebc53ac68f service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:09.286983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1043/2065] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:09.291628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1023/2066] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:09 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:12:09.404496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d783e76-27c2-413c-9b98-6882cfbca30b service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:09.413971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d783e76-27c2-413c-9b98-6882cfbca30b service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:09.417773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50084cb-aa8a-432f-ba79-d5bf9824d6c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:09.418361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50084cb-aa8a-432f-ba79-d5bf9824d6c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:09.418634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50084cb-aa8a-432f-ba79-d5bf9824d6c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:09.418901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50084cb-aa8a-432f-ba79-d5bf9824d6c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:09.481148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50084cb-aa8a-432f-ba79-d5bf9824d6c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:09.482677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50084cb-aa8a-432f-ba79-d5bf9824d6c6 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:09.493195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1044/2067] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:09.762372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d530907-5a50-4bb0-809c-25ebc53ac68f service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:09.875096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d530907-5a50-4bb0-809c-25ebc53ac68f service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:09.879699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:09.880405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:09.880679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:09.880952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] PATH_INFO: `/v3/users/334bb3ffcafb40348522a25f1f8357dc` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:09.882240 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=334bb3ffcafb40348522a25f1f8357dc)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:09.884397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:09.932689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334bb3ffcafb40348522a25f1f8357dc'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:09.933364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334bb3ffcafb40348522a25f1f8357dc'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:09.939971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334bb3ffcafb40348522a25f1f8357dc'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:09.940590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334bb3ffcafb40348522a25f1f8357dc'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:09.951961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334bb3ffcafb40348522a25f1f8357dc'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:09.964515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1024/2068] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:09 2022] DELETE /identity/v3/users/334bb3ffcafb40348522a25f1f8357dc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:09.969059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8534a-5191-4bc3-bf0d-d0fb1eb709a0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:09.969378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8534a-5191-4bc3-bf0d-d0fb1eb709a0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:09.969656 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8534a-5191-4bc3-bf0d-d0fb1eb709a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:09.989185 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c7c8534a-5191-4bc3-bf0d-d0fb1eb709a0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:09.990391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c7c8534a-5191-4bc3-bf0d-d0fb1eb709a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:10.048062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1045/2069] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:10.058887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:10.060374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:10.066061 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:10.121754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a15394-9f2f-4024-b672-fc764d9fdd6c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:10.125476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976c23d-6f05-4449-8e93-863189bb2986 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:10.126082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976c23d-6f05-4449-8e93-863189bb2986 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:10.126357 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976c23d-6f05-4449-8e93-863189bb2986 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:10.126623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976c23d-6f05-4449-8e93-863189bb2986 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:10.136517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976c23d-6f05-4449-8e93-863189bb2986 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:10.137967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976c23d-6f05-4449-8e93-863189bb2986 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:10.147903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1025/2070] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:10.503027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8534a-5191-4bc3-bf0d-d0fb1eb709a0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:10.504683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c7c8534a-5191-4bc3-bf0d-d0fb1eb709a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:10.524108 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8534a-5191-4bc3-bf0d-d0fb1eb709a0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:10.529966 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:10.530610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:10.530980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:10.531143 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] PATH_INFO: `/v3/projects/44a035550b2d402cbb7f87a031a6d5b4` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:10.532414 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44a035550b2d402cbb7f87a031a6d5b4)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:10.534685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:10.586243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44a035550b2d402cbb7f87a031a6d5b4', 'request_id': 'req-b1bf745e-e674-433f-8e64-74d873bbf5c7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:10.593213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44a035550b2d402cbb7f87a031a6d5b4', 'request_id': 'req-b1bf745e-e674-433f-8e64-74d873bbf5c7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:10.594490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:10.599569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44a035550b2d402cbb7f87a031a6d5b4', 'request_id': 'req-b1bf745e-e674-433f-8e64-74d873bbf5c7'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:10.614573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] The token cache is being invalidate because project 44a035550b2d402cbb7f87a031a6d5b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:10.614706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:10.615366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:10.618314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1046/2071] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:10 2022] DELETE /identity/v3/projects/44a035550b2d402cbb7f87a031a6d5b4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:10.622446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976c23d-6f05-4449-8e93-863189bb2986 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:10.633603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976c23d-6f05-4449-8e93-863189bb2986 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:10.691824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c4fca-7425-4599-a0c2-087536c5ad78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:10.692446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c4fca-7425-4599-a0c2-087536c5ad78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:10.692724 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c4fca-7425-4599-a0c2-087536c5ad78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:10.692997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c4fca-7425-4599-a0c2-087536c5ad78 admin admin] PATH_INFO: `/v3/users/ce9210ed36d4453bb5d7a79eae236ca8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:10.694700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c4fca-7425-4599-a0c2-087536c5ad78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9210ed36d4453bb5d7a79eae236ca8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:10.696672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c4fca-7425-4599-a0c2-087536c5ad78 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:10.741772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-139c4fca-7425-4599-a0c2-087536c5ad78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9210ed36d4453bb5d7a79eae236ca8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:10.742413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-139c4fca-7425-4599-a0c2-087536c5ad78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9210ed36d4453bb5d7a79eae236ca8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:10.749063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-139c4fca-7425-4599-a0c2-087536c5ad78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9210ed36d4453bb5d7a79eae236ca8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:10.749715 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-139c4fca-7425-4599-a0c2-087536c5ad78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9210ed36d4453bb5d7a79eae236ca8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:10.761642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-139c4fca-7425-4599-a0c2-087536c5ad78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9210ed36d4453bb5d7a79eae236ca8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:10.774384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1026/2072] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:10 2022] DELETE /identity/v3/users/ce9210ed36d4453bb5d7a79eae236ca8 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:11.168889 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:11.170412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:11.184299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf745e-e674-433f-8e64-74d873bbf5c7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:11.241484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:11.242105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:11.242379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:11.242648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] PATH_INFO: `/v3/projects/29810d29944a44ef8e9d4c4dba7e49cc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:11.243746 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29810d29944a44ef8e9d4c4dba7e49cc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:11.245662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:11.291220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29810d29944a44ef8e9d4c4dba7e49cc', 'request_id': 'req-62aa4e29-7538-4cad-86f7-11f487e2b7a9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:11.297078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29810d29944a44ef8e9d4c4dba7e49cc', 'request_id': 'req-62aa4e29-7538-4cad-86f7-11f487e2b7a9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:11.302726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29810d29944a44ef8e9d4c4dba7e49cc', 'request_id': 'req-62aa4e29-7538-4cad-86f7-11f487e2b7a9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:11.314706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] The token cache is being invalidate because project 29810d29944a44ef8e9d4c4dba7e49cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:11.314989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:11.315608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-62aa4e29-7538-4cad-86f7-11f487e2b7a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:11.317063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1047/2073] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:11 2022] DELETE /identity/v3/projects/29810d29944a44ef8e9d4c4dba7e49cc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:11.322428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b660b5d-9070-41b4-b8db-490f1831c05f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:11.322708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b660b5d-9070-41b4-b8db-490f1831c05f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:11.322976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b660b5d-9070-41b4-b8db-490f1831c05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:11.342643 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7b660b5d-9070-41b4-b8db-490f1831c05f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:11.343821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7b660b5d-9070-41b4-b8db-490f1831c05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:11.363557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:11.406944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1027/2074] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:11.415609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54999926-8ba6-4f7d-b0bc-3e9894c9d121 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:11.415897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54999926-8ba6-4f7d-b0bc-3e9894c9d121 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:11.416173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54999926-8ba6-4f7d-b0bc-3e9894c9d121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:11.437410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-54999926-8ba6-4f7d-b0bc-3e9894c9d121 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:11.438622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-54999926-8ba6-4f7d-b0bc-3e9894c9d121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:11.447018 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1048/2075] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:11.451212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b660b5d-9070-41b4-b8db-490f1831c05f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:11.461966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b660b5d-9070-41b4-b8db-490f1831c05f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:11.466680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f031e45-a2be-4b69-9a1d-bb7c6461beda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:11.467269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f031e45-a2be-4b69-9a1d-bb7c6461beda admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:11.467546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f031e45-a2be-4b69-9a1d-bb7c6461beda admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:11.467929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f031e45-a2be-4b69-9a1d-bb7c6461beda admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:11.468593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f031e45-a2be-4b69-9a1d-bb7c6461beda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:12:11.468894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f031e45-a2be-4b69-9a1d-bb7c6461beda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:11.470884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f031e45-a2be-4b69-9a1d-bb7c6461beda admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:11.483508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1028/2076] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:12:11.490222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54999926-8ba6-4f7d-b0bc-3e9894c9d121 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:11.500279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54999926-8ba6-4f7d-b0bc-3e9894c9d121 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:11.505243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea6aba-e035-42ef-8747-e9beef2b129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:11.505691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea6aba-e035-42ef-8747-e9beef2b129d admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:11.505969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea6aba-e035-42ef-8747-e9beef2b129d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:11.506768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea6aba-e035-42ef-8747-e9beef2b129d admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:11.507410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea6aba-e035-42ef-8747-e9beef2b129d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:11.509552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea6aba-e035-42ef-8747-e9beef2b129d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:11.539941 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1049/2077] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:11 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:11.543981 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f031e45-a2be-4b69-9a1d-bb7c6461beda admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:11.555062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f031e45-a2be-4b69-9a1d-bb7c6461beda admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:11.610225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271f2c8-cf90-4b6c-a4bf-5e47d81749e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:11.610818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271f2c8-cf90-4b6c-a4bf-5e47d81749e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:11.611095 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271f2c8-cf90-4b6c-a4bf-5e47d81749e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:11.611365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9271f2c8-cf90-4b6c-a4bf-5e47d81749e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:11.611985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271f2c8-cf90-4b6c-a4bf-5e47d81749e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:11.613987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9271f2c8-cf90-4b6c-a4bf-5e47d81749e0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:11.636599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1029/2078] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:11 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:11.640911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea6aba-e035-42ef-8747-e9beef2b129d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:11.651640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea6aba-e035-42ef-8747-e9beef2b129d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:11.656271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a4c66-ffd7-4327-af9d-e13e8b9e5a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:11.656867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a4c66-ffd7-4327-af9d-e13e8b9e5a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:11.657219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a4c66-ffd7-4327-af9d-e13e8b9e5a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:11.657450 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a4c66-ffd7-4327-af9d-e13e8b9e5a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:11.658096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a4c66-ffd7-4327-af9d-e13e8b9e5a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:11.659670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a4c66-ffd7-4327-af9d-e13e8b9e5a5a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:11.669827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1050/2079] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:11.673730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271f2c8-cf90-4b6c-a4bf-5e47d81749e0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:11.684507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9271f2c8-cf90-4b6c-a4bf-5e47d81749e0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:11.689170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d163df-e373-4dd9-a154-1064952af0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:11.689757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d163df-e373-4dd9-a154-1064952af0c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:11.690033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d163df-e373-4dd9-a154-1064952af0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:11.690301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d163df-e373-4dd9-a154-1064952af0c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:11.691063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d163df-e373-4dd9-a154-1064952af0c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:11.692639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d163df-e373-4dd9-a154-1064952af0c5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:11.703622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1030/2080] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:11.707706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a4c66-ffd7-4327-af9d-e13e8b9e5a5a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:11.719583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a4c66-ffd7-4327-af9d-e13e8b9e5a5a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:11.724448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b851f94-673f-4438-9af7-d8b950da0979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:11.725493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b851f94-673f-4438-9af7-d8b950da0979 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:11.725810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b851f94-673f-4438-9af7-d8b950da0979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:11.726258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b851f94-673f-4438-9af7-d8b950da0979 admin admin] PATH_INFO: `/v3/projects/bb7c0295599446618409040c2f0f47d9/users/425a2609f8744b77bc7131eba51b579f/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:11.742402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b851f94-673f-4438-9af7-d8b950da0979 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb7c0295599446618409040c2f0f47d9, user_id=425a2609f8744b77bc7131eba51b579f, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:11.744720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b851f94-673f-4438-9af7-d8b950da0979 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:11.759119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1051/2081] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:11 2022] PUT /identity/v3/projects/bb7c0295599446618409040c2f0f47d9/users/425a2609f8744b77bc7131eba51b579f/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:11.769388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d8f71-ee8b-4fda-954d-7f8bb32e997a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:11.769675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d8f71-ee8b-4fda-954d-7f8bb32e997a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:11.769953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d8f71-ee8b-4fda-954d-7f8bb32e997a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:11.799242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c71d8f71-ee8b-4fda-954d-7f8bb32e997a None None] MFA Rules not processed for user `425a2609f8744b77bc7131eba51b579f`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:11.800452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c71d8f71-ee8b-4fda-954d-7f8bb32e997a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:11.866969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1031/2082] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:11.871471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b851f94-673f-4438-9af7-d8b950da0979 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:11.882724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:11.941725 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b851f94-673f-4438-9af7-d8b950da0979 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:11.946775 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39e433-d4e4-451c-85a2-affd18511131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:11.947379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39e433-d4e4-451c-85a2-affd18511131 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:11.947803 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39e433-d4e4-451c-85a2-affd18511131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:11.948079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39e433-d4e4-451c-85a2-affd18511131 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:11.948714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39e433-d4e4-451c-85a2-affd18511131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:11.950632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39e433-d4e4-451c-85a2-affd18511131 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:11.982523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1052/2083] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:11 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:11.983841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d8f71-ee8b-4fda-954d-7f8bb32e997a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:11.996344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d8f71-ee8b-4fda-954d-7f8bb32e997a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:12.015448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:12.060605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71a44-d8f7-4ac4-9561-e615e79003ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:12.063748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71a44-d8f7-4ac4-9561-e615e79003ce admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:12.063748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71a44-d8f7-4ac4-9561-e615e79003ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:12.063748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd71a44-d8f7-4ac4-9561-e615e79003ce admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:12.063748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd71a44-d8f7-4ac4-9561-e615e79003ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:12.065224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd71a44-d8f7-4ac4-9561-e615e79003ce admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:12.093505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1032/2084] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:11 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:12.095040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39e433-d4e4-451c-85a2-affd18511131 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:12.108361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39e433-d4e4-451c-85a2-affd18511131 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:12.113565 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb94e1-88cf-4a69-bf27-d6b731a56160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:12.114274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb94e1-88cf-4a69-bf27-d6b731a56160 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:12.114624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb94e1-88cf-4a69-bf27-d6b731a56160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:12.114964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadb94e1-88cf-4a69-bf27-d6b731a56160 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:12.116315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb94e1-88cf-4a69-bf27-d6b731a56160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:12.118064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadb94e1-88cf-4a69-bf27-d6b731a56160 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:12.132007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1053/2085] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:12.137387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71a44-d8f7-4ac4-9561-e615e79003ce admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:12.149490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd71a44-d8f7-4ac4-9561-e615e79003ce admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:12.154375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b5af7-267d-492c-aaca-cff42d399ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:12.155153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b5af7-267d-492c-aaca-cff42d399ac8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:12.155599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b5af7-267d-492c-aaca-cff42d399ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:12.155975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b5af7-267d-492c-aaca-cff42d399ac8 admin admin] PATH_INFO: `/v3/projects/d814456ee36d428d95bc0be3d51908b3/users/481a9c88ee6148bcad6238b1b7269373/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:12.173213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b5af7-267d-492c-aaca-cff42d399ac8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d814456ee36d428d95bc0be3d51908b3, user_id=481a9c88ee6148bcad6238b1b7269373, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:12.175609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b5af7-267d-492c-aaca-cff42d399ac8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:12.190925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1033/2086] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:12 2022] PUT /identity/v3/projects/d814456ee36d428d95bc0be3d51908b3/users/481a9c88ee6148bcad6238b1b7269373/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:12.195535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb94e1-88cf-4a69-bf27-d6b731a56160 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:12.206209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadb94e1-88cf-4a69-bf27-d6b731a56160 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:12.267754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8d1d5-3c74-4d87-b168-33f10631f1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:12.268459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8d1d5-3c74-4d87-b168-33f10631f1bb admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:12.268739 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8d1d5-3c74-4d87-b168-33f10631f1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:12.269012 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8d1d5-3c74-4d87-b168-33f10631f1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:12.269740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8d1d5-3c74-4d87-b168-33f10631f1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:12.271367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8d1d5-3c74-4d87-b168-33f10631f1bb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:12.285394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1054/2087] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:12:12.286603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b5af7-267d-492c-aaca-cff42d399ac8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:12.297934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b5af7-267d-492c-aaca-cff42d399ac8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:12.303583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2f0f2-49f9-482b-8ed7-8f317b5dd42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:12.304254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2f0f2-49f9-482b-8ed7-8f317b5dd42f admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:12.304531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2f0f2-49f9-482b-8ed7-8f317b5dd42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:12.304802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a2f0f2-49f9-482b-8ed7-8f317b5dd42f admin admin] PATH_INFO: `/v3/projects/d814456ee36d428d95bc0be3d51908b3/users/481a9c88ee6148bcad6238b1b7269373/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:12.306792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2f0f2-49f9-482b-8ed7-8f317b5dd42f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d814456ee36d428d95bc0be3d51908b3, user_id=481a9c88ee6148bcad6238b1b7269373, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:12.309465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a2f0f2-49f9-482b-8ed7-8f317b5dd42f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:12.320768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1034/2088] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:12 2022] PUT /identity/v3/projects/d814456ee36d428d95bc0be3d51908b3/users/481a9c88ee6148bcad6238b1b7269373/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:12.332039 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7b42-b696-4b8b-bd2a-f50082f4cc6e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:12.332343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7b42-b696-4b8b-bd2a-f50082f4cc6e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:12.332603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf7b42-b696-4b8b-bd2a-f50082f4cc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:12.362685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-20cf7b42-b696-4b8b-bd2a-f50082f4cc6e None None] MFA Rules not processed for user `481a9c88ee6148bcad6238b1b7269373`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:12.363857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-20cf7b42-b696-4b8b-bd2a-f50082f4cc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:12.443978 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1055/2089] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:12 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:12.455667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2f0f2-49f9-482b-8ed7-8f317b5dd42f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:12.457667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-69a2f0f2-49f9-482b-8ed7-8f317b5dd42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:12.518185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a2f0f2-49f9-482b-8ed7-8f317b5dd42f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:12.523256 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc19bdb-d68f-478e-aed0-6545606f36e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:12.524110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc19bdb-d68f-478e-aed0-6545606f36e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:12.524484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc19bdb-d68f-478e-aed0-6545606f36e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:12.524853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc19bdb-d68f-478e-aed0-6545606f36e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:12.534853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc19bdb-d68f-478e-aed0-6545606f36e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:12.536280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc19bdb-d68f-478e-aed0-6545606f36e1 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:12.547525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1035/2090] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:12 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:12:12.578418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf7b42-b696-4b8b-bd2a-f50082f4cc6e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:12.585958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:12.656406 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf7b42-b696-4b8b-bd2a-f50082f4cc6e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:12.661457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:12.661888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:12.662212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:12.663030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] PATH_INFO: `/v3/users/425a2609f8744b77bc7131eba51b579f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:12.664213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] RBAC: Authorizing `identity:delete_user(user_id=425a2609f8744b77bc7131eba51b579f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:12.666799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:12.729654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '425a2609f8744b77bc7131eba51b579f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:12.730886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '425a2609f8744b77bc7131eba51b579f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:12.737446 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '425a2609f8744b77bc7131eba51b579f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:12.765634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '425a2609f8744b77bc7131eba51b579f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:12.777819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '425a2609f8744b77bc7131eba51b579f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:12.800348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1056/2091] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:12 2022] DELETE /identity/v3/users/425a2609f8744b77bc7131eba51b579f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:12.805147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4577ead-a614-45bf-af55-d4f095cb6f0e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:12.806025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4577ead-a614-45bf-af55-d4f095cb6f0e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:12.806374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4577ead-a614-45bf-af55-d4f095cb6f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:12.833343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c4577ead-a614-45bf-af55-d4f095cb6f0e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:12.835302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c4577ead-a614-45bf-af55-d4f095cb6f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:12.898056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1036/2092] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:12.907588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:12.908978 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:12.969661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b9b1c-2aff-407d-bdac-b00965aa74de admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:12.974203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd030623-4599-478b-9a99-1ed6704ffab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:12.975520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd030623-4599-478b-9a99-1ed6704ffab6 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:12.976179 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd030623-4599-478b-9a99-1ed6704ffab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:12.976752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd030623-4599-478b-9a99-1ed6704ffab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:12.991772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd030623-4599-478b-9a99-1ed6704ffab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:12.993611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd030623-4599-478b-9a99-1ed6704ffab6 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:13.008600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1057/2093] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:13.301607 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4577ead-a614-45bf-af55-d4f095cb6f0e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:13.304007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c4577ead-a614-45bf-af55-d4f095cb6f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:13.326011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4577ead-a614-45bf-af55-d4f095cb6f0e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:13.339318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:13.339969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:13.340249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:13.340650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] PATH_INFO: `/v3/projects/bb7c0295599446618409040c2f0f47d9` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:13.342090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb7c0295599446618409040c2f0f47d9)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:13.344364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:13.363480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:13.403949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb7c0295599446618409040c2f0f47d9', 'request_id': 'req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:13.411168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb7c0295599446618409040c2f0f47d9', 'request_id': 'req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:13.417799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb7c0295599446618409040c2f0f47d9', 'request_id': 'req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:13.429340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] The token cache is being invalidate because project bb7c0295599446618409040c2f0f47d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:13.429628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:13.430269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:13.431735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1037/2094] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:13 2022] DELETE /identity/v3/projects/bb7c0295599446618409040c2f0f47d9 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:13.435669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd030623-4599-478b-9a99-1ed6704ffab6 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:13.446069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd030623-4599-478b-9a99-1ed6704ffab6 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:13.468976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:13.503177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c6deb-d687-48a5-88d6-3af55a87aa10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:13.503833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c6deb-d687-48a5-88d6-3af55a87aa10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:13.504113 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c6deb-d687-48a5-88d6-3af55a87aa10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:13.504383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c6deb-d687-48a5-88d6-3af55a87aa10 admin admin] PATH_INFO: `/v3/users/481a9c88ee6148bcad6238b1b7269373` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:13.505531 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c6deb-d687-48a5-88d6-3af55a87aa10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=481a9c88ee6148bcad6238b1b7269373)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:13.507421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44c6deb-d687-48a5-88d6-3af55a87aa10 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:13.554907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e44c6deb-d687-48a5-88d6-3af55a87aa10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481a9c88ee6148bcad6238b1b7269373'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:13.555718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e44c6deb-d687-48a5-88d6-3af55a87aa10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481a9c88ee6148bcad6238b1b7269373'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:13.556333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e44c6deb-d687-48a5-88d6-3af55a87aa10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481a9c88ee6148bcad6238b1b7269373'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:13.569160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e44c6deb-d687-48a5-88d6-3af55a87aa10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481a9c88ee6148bcad6238b1b7269373'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:13.576604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-e44c6deb-d687-48a5-88d6-3af55a87aa10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481a9c88ee6148bcad6238b1b7269373'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:13.589853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1058/2095] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:13 2022] DELETE /identity/v3/users/481a9c88ee6148bcad6238b1b7269373 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:13.913428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:13.914986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:13.933216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4132eb4-f77c-45a8-bebb-1b4c945c5de3 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:13.988398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:13.988969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:13.989297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:13.989648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] PATH_INFO: `/v3/projects/d814456ee36d428d95bc0be3d51908b3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:13.991918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d814456ee36d428d95bc0be3d51908b3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:13.994272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:14.043202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd814456ee36d428d95bc0be3d51908b3', 'request_id': 'req-c305a25d-f70d-4b9a-a4d4-5e4959e91151'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:14.050743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd814456ee36d428d95bc0be3d51908b3', 'request_id': 'req-c305a25d-f70d-4b9a-a4d4-5e4959e91151'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:14.059170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd814456ee36d428d95bc0be3d51908b3', 'request_id': 'req-c305a25d-f70d-4b9a-a4d4-5e4959e91151'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:14.076351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] The token cache is being invalidate because project d814456ee36d428d95bc0be3d51908b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:14.076580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:14.077278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c305a25d-f70d-4b9a-a4d4-5e4959e91151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:14.078799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1038/2096] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:13 2022] DELETE /identity/v3/projects/d814456ee36d428d95bc0be3d51908b3 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:14.083532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d1f09-4f81-40f1-be9e-4c09828fd0d7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.083826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d1f09-4f81-40f1-be9e-4c09828fd0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.084168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d1f09-4f81-40f1-be9e-4c09828fd0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.105132 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-4f3d1f09-4f81-40f1-be9e-4c09828fd0d7 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:14.106386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4f3d1f09-4f81-40f1-be9e-4c09828fd0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:14.175490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1059/2097] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:14.184568 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ba927-48cf-47bd-b885-08698f85914e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.184858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ba927-48cf-47bd-b885-08698f85914e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.185163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ba927-48cf-47bd-b885-08698f85914e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.205944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-311ba927-48cf-47bd-b885-08698f85914e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:14.207156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-311ba927-48cf-47bd-b885-08698f85914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:14.215063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1039/2098] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:14.219596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d1f09-4f81-40f1-be9e-4c09828fd0d7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:14.231131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d1f09-4f81-40f1-be9e-4c09828fd0d7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:14.235865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d17ed3-5e76-4df0-b9cf-97d987de2680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:14.236474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d17ed3-5e76-4df0-b9cf-97d987de2680 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.236781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d17ed3-5e76-4df0-b9cf-97d987de2680 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.237059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d17ed3-5e76-4df0-b9cf-97d987de2680 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.237778 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d17ed3-5e76-4df0-b9cf-97d987de2680 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:12:14.238054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d17ed3-5e76-4df0-b9cf-97d987de2680 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:14.239701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d17ed3-5e76-4df0-b9cf-97d987de2680 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:14.252940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1060/2099] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:12:14.257835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ba927-48cf-47bd-b885-08698f85914e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:14.267618 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ba927-48cf-47bd-b885-08698f85914e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:14.272347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0935da-4b85-4b55-b2a4-cf3510fde977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:14.272970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0935da-4b85-4b55-b2a4-cf3510fde977 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.273268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0935da-4b85-4b55-b2a4-cf3510fde977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.273539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0935da-4b85-4b55-b2a4-cf3510fde977 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.274167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0935da-4b85-4b55-b2a4-cf3510fde977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:14.276050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0935da-4b85-4b55-b2a4-cf3510fde977 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:14.307175 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:14.317276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1040/2100] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:14 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:14.319237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d17ed3-5e76-4df0-b9cf-97d987de2680 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:14.329398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d17ed3-5e76-4df0-b9cf-97d987de2680 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:14.367617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:14.405617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c527-da4c-4c3b-a3ee-618bcd0d11a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:14.413977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c527-da4c-4c3b-a3ee-618bcd0d11a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.413977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c527-da4c-4c3b-a3ee-618bcd0d11a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.413977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0c527-da4c-4c3b-a3ee-618bcd0d11a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.413977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0c527-da4c-4c3b-a3ee-618bcd0d11a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:14.413977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0c527-da4c-4c3b-a3ee-618bcd0d11a7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:14.434439 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1061/2101] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:14 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:14.435713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0935da-4b85-4b55-b2a4-cf3510fde977 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:14.446573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0935da-4b85-4b55-b2a4-cf3510fde977 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:14.451342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc00251-d6ea-4b45-bd7b-1d64cc3da296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:14.451935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc00251-d6ea-4b45-bd7b-1d64cc3da296 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.452214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc00251-d6ea-4b45-bd7b-1d64cc3da296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.452486 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc00251-d6ea-4b45-bd7b-1d64cc3da296 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.453154 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc00251-d6ea-4b45-bd7b-1d64cc3da296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:14.454743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc00251-d6ea-4b45-bd7b-1d64cc3da296 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:14.466043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1041/2102] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:14.470084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c527-da4c-4c3b-a3ee-618bcd0d11a7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:14.481425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0c527-da4c-4c3b-a3ee-618bcd0d11a7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:14.486072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed431be6-2c6c-43a5-8317-c58d74fb0a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:14.486663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed431be6-2c6c-43a5-8317-c58d74fb0a5e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.486949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed431be6-2c6c-43a5-8317-c58d74fb0a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.487221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed431be6-2c6c-43a5-8317-c58d74fb0a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.487864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed431be6-2c6c-43a5-8317-c58d74fb0a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:14.489505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed431be6-2c6c-43a5-8317-c58d74fb0a5e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:14.500598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1062/2103] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:14.504879 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc00251-d6ea-4b45-bd7b-1d64cc3da296 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:14.515462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc00251-d6ea-4b45-bd7b-1d64cc3da296 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:14.520747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cbc83-ee9e-4280-bda6-b12ff4701007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:14.521583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cbc83-ee9e-4280-bda6-b12ff4701007 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.522272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cbc83-ee9e-4280-bda6-b12ff4701007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.522622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644cbc83-ee9e-4280-bda6-b12ff4701007 admin admin] PATH_INFO: `/v3/projects/625b87294d5046e69e7513ab0d81ac00/users/d671407b7c7541b489b4e3c9fbafe3ad/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.539402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cbc83-ee9e-4280-bda6-b12ff4701007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=625b87294d5046e69e7513ab0d81ac00, user_id=d671407b7c7541b489b4e3c9fbafe3ad, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:14.542179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644cbc83-ee9e-4280-bda6-b12ff4701007 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:14.555361 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1042/2104] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:14 2022] PUT /identity/v3/projects/625b87294d5046e69e7513ab0d81ac00/users/d671407b7c7541b489b4e3c9fbafe3ad/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:14.565590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b139d41-6b7d-4adc-ad8b-c0c3bb187288 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.565870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b139d41-6b7d-4adc-ad8b-c0c3bb187288 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.566141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b139d41-6b7d-4adc-ad8b-c0c3bb187288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.594660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6b139d41-6b7d-4adc-ad8b-c0c3bb187288 None None] MFA Rules not processed for user `d671407b7c7541b489b4e3c9fbafe3ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:14.595890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6b139d41-6b7d-4adc-ad8b-c0c3bb187288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:14.661563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1063/2105] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:14.666845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cbc83-ee9e-4280-bda6-b12ff4701007 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:14.732028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644cbc83-ee9e-4280-bda6-b12ff4701007 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:14.736739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19767857-9993-431b-b6ad-8cedd1774c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:14.737398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19767857-9993-431b-b6ad-8cedd1774c86 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.737681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19767857-9993-431b-b6ad-8cedd1774c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.737954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19767857-9993-431b-b6ad-8cedd1774c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.738597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19767857-9993-431b-b6ad-8cedd1774c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:14.741017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19767857-9993-431b-b6ad-8cedd1774c86 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:14.769569 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1043/2106] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:14 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:14.774554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b139d41-6b7d-4adc-ad8b-c0c3bb187288 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:14.785706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b139d41-6b7d-4adc-ad8b-c0c3bb187288 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:14.844569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44adc5f-d94f-4e8b-9644-31afe88898b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:14.845210 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44adc5f-d94f-4e8b-9644-31afe88898b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.845485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44adc5f-d94f-4e8b-9644-31afe88898b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.845754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44adc5f-d94f-4e8b-9644-31afe88898b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.846481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44adc5f-d94f-4e8b-9644-31afe88898b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:14.848485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44adc5f-d94f-4e8b-9644-31afe88898b8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:14.872045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1064/2107] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:14 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:14.875922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19767857-9993-431b-b6ad-8cedd1774c86 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:14.886779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19767857-9993-431b-b6ad-8cedd1774c86 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:14.891375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74df09-d1db-4fb5-ac40-cac7d6ab7490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:14.891954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74df09-d1db-4fb5-ac40-cac7d6ab7490 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.892227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74df09-d1db-4fb5-ac40-cac7d6ab7490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.892539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca74df09-d1db-4fb5-ac40-cac7d6ab7490 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.893468 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74df09-d1db-4fb5-ac40-cac7d6ab7490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:14.895654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca74df09-d1db-4fb5-ac40-cac7d6ab7490 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:14.908574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1044/2108] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:14.912795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44adc5f-d94f-4e8b-9644-31afe88898b8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:14.924540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44adc5f-d94f-4e8b-9644-31afe88898b8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:14.930509 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67b162-3054-4b4f-a676-2f33329afabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:14.932206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67b162-3054-4b4f-a676-2f33329afabc admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:14.932206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67b162-3054-4b4f-a676-2f33329afabc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:14.932497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a67b162-3054-4b4f-a676-2f33329afabc admin admin] PATH_INFO: `/v3/projects/b6c9127cf9a447bb958835891d053c52/users/ef84b3128a0c45978215e7d5583eeacf/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:14.951121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67b162-3054-4b4f-a676-2f33329afabc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6c9127cf9a447bb958835891d053c52, user_id=ef84b3128a0c45978215e7d5583eeacf, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:14.953995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a67b162-3054-4b4f-a676-2f33329afabc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:14.972521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1065/2109] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:14 2022] PUT /identity/v3/projects/b6c9127cf9a447bb958835891d053c52/users/ef84b3128a0c45978215e7d5583eeacf/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:14.977138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74df09-d1db-4fb5-ac40-cac7d6ab7490 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:14.992699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca74df09-d1db-4fb5-ac40-cac7d6ab7490 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:15.065495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67726ed5-60cc-4b83-aefa-cff9b59a3d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:15.066108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67726ed5-60cc-4b83-aefa-cff9b59a3d4c admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:15.066388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67726ed5-60cc-4b83-aefa-cff9b59a3d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:15.066660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67726ed5-60cc-4b83-aefa-cff9b59a3d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:15.067311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67726ed5-60cc-4b83-aefa-cff9b59a3d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:15.070710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67726ed5-60cc-4b83-aefa-cff9b59a3d4c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:15.088934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1045/2110] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:15.093091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67b162-3054-4b4f-a676-2f33329afabc admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:15.104556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a67b162-3054-4b4f-a676-2f33329afabc admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:15.109717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c5ee1-c19c-4bdf-a1a7-31471f267be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:15.110384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c5ee1-c19c-4bdf-a1a7-31471f267be2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:15.110663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c5ee1-c19c-4bdf-a1a7-31471f267be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:15.110936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c5ee1-c19c-4bdf-a1a7-31471f267be2 admin admin] PATH_INFO: `/v3/projects/b6c9127cf9a447bb958835891d053c52/users/ef84b3128a0c45978215e7d5583eeacf/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:15.112888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c5ee1-c19c-4bdf-a1a7-31471f267be2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6c9127cf9a447bb958835891d053c52, user_id=ef84b3128a0c45978215e7d5583eeacf, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:15.116366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c5ee1-c19c-4bdf-a1a7-31471f267be2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:15.118984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:15.129145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1066/2111] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:15 2022] PUT /identity/v3/projects/b6c9127cf9a447bb958835891d053c52/users/ef84b3128a0c45978215e7d5583eeacf/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:15.140217 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cff7e-8164-4727-9a0e-6106cc8a8227 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:15.140506 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cff7e-8164-4727-9a0e-6106cc8a8227 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:15.140786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566cff7e-8164-4727-9a0e-6106cc8a8227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:15.168822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-566cff7e-8164-4727-9a0e-6106cc8a8227 None None] MFA Rules not processed for user `ef84b3128a0c45978215e7d5583eeacf`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:15.170097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-566cff7e-8164-4727-9a0e-6106cc8a8227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:15.195118 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:15.243056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1046/2112] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:15 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:15.253643 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c5ee1-c19c-4bdf-a1a7-31471f267be2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:15.255342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-620c5ee1-c19c-4bdf-a1a7-31471f267be2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:15.313815 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c5ee1-c19c-4bdf-a1a7-31471f267be2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:15.317374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05b9d0-42e8-4747-826e-93afdcea2a8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:15.318000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05b9d0-42e8-4747-826e-93afdcea2a8d service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:15.318418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05b9d0-42e8-4747-826e-93afdcea2a8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:15.318689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05b9d0-42e8-4747-826e-93afdcea2a8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:15.327538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05b9d0-42e8-4747-826e-93afdcea2a8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:15.328945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05b9d0-42e8-4747-826e-93afdcea2a8d service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:15.339120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1067/2113] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:16.414303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cff7e-8164-4727-9a0e-6106cc8a8227 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:16.503168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566cff7e-8164-4727-9a0e-6106cc8a8227 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:16.508619 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:16.509319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:16.509602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:16.509932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] PATH_INFO: `/v3/users/d671407b7c7541b489b4e3c9fbafe3ad` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:16.522197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d671407b7c7541b489b4e3c9fbafe3ad)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:16.524186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:16.594796 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd671407b7c7541b489b4e3c9fbafe3ad'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:16.595880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd671407b7c7541b489b4e3c9fbafe3ad'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:16.612284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd671407b7c7541b489b4e3c9fbafe3ad'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:16.613140 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd671407b7c7541b489b4e3c9fbafe3ad'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:16.626535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd671407b7c7541b489b4e3c9fbafe3ad'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:16.641515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1047/2114] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:16 2022] DELETE /identity/v3/users/d671407b7c7541b489b4e3c9fbafe3ad => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:16.646142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf188576-29e2-4ace-bbed-f3226dd3aeef None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:16.647274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf188576-29e2-4ace-bbed-f3226dd3aeef None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:16.647274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf188576-29e2-4ace-bbed-f3226dd3aeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:16.666462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cf188576-29e2-4ace-bbed-f3226dd3aeef None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:16.667658 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cf188576-29e2-4ace-bbed-f3226dd3aeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:16.754255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1068/2115] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:16.765333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:16.767151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:16.822773 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:16.841324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec04c9-6e57-4893-b493-c9c620204656 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:16.845231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35625a22-4efe-413f-b6a5-68676f3dd840 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:16.845833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625a22-4efe-413f-b6a5-68676f3dd840 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:16.846109 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625a22-4efe-413f-b6a5-68676f3dd840 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:16.846376 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35625a22-4efe-413f-b6a5-68676f3dd840 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:16.857781 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35625a22-4efe-413f-b6a5-68676f3dd840 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:16.859059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35625a22-4efe-413f-b6a5-68676f3dd840 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:16.870856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1048/2116] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:17.177730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf188576-29e2-4ace-bbed-f3226dd3aeef None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:17.179378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cf188576-29e2-4ace-bbed-f3226dd3aeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:17.193070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf188576-29e2-4ace-bbed-f3226dd3aeef None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:17.198715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:17.199429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:17.199772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:17.200115 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] PATH_INFO: `/v3/projects/625b87294d5046e69e7513ab0d81ac00` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:17.201416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=625b87294d5046e69e7513ab0d81ac00)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:17.203125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:17.252681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '625b87294d5046e69e7513ab0d81ac00', 'request_id': 'req-fc244f1e-231d-4f7a-bb4c-38cd891f7441'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:17.259647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '625b87294d5046e69e7513ab0d81ac00', 'request_id': 'req-fc244f1e-231d-4f7a-bb4c-38cd891f7441'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:17.265872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '625b87294d5046e69e7513ab0d81ac00', 'request_id': 'req-fc244f1e-231d-4f7a-bb4c-38cd891f7441'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:17.280188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] The token cache is being invalidate because project 625b87294d5046e69e7513ab0d81ac00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:17.280505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:17.281294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:17.283220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1069/2117] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:17 2022] DELETE /identity/v3/projects/625b87294d5046e69e7513ab0d81ac00 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:17.288347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35625a22-4efe-413f-b6a5-68676f3dd840 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:17.297681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35625a22-4efe-413f-b6a5-68676f3dd840 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:17.359435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e75bd3-104b-468d-9ad6-c20ce5e53e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:17.360073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e75bd3-104b-468d-9ad6-c20ce5e53e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:17.360520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e75bd3-104b-468d-9ad6-c20ce5e53e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:17.360803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e75bd3-104b-468d-9ad6-c20ce5e53e72 admin admin] PATH_INFO: `/v3/users/ef84b3128a0c45978215e7d5583eeacf` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:17.363514 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e75bd3-104b-468d-9ad6-c20ce5e53e72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef84b3128a0c45978215e7d5583eeacf)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:17.366232 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e75bd3-104b-468d-9ad6-c20ce5e53e72 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:17.444826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-05e75bd3-104b-468d-9ad6-c20ce5e53e72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef84b3128a0c45978215e7d5583eeacf'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:17.444826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-05e75bd3-104b-468d-9ad6-c20ce5e53e72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef84b3128a0c45978215e7d5583eeacf'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:17.453883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-05e75bd3-104b-468d-9ad6-c20ce5e53e72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef84b3128a0c45978215e7d5583eeacf'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:17.455434 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-05e75bd3-104b-468d-9ad6-c20ce5e53e72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef84b3128a0c45978215e7d5583eeacf'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:17.484415 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-05e75bd3-104b-468d-9ad6-c20ce5e53e72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef84b3128a0c45978215e7d5583eeacf'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:17.505334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1049/2118] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:17 2022] DELETE /identity/v3/users/ef84b3128a0c45978215e7d5583eeacf => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:17.855696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:17.857225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:17.872798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc244f1e-231d-4f7a-bb4c-38cd891f7441 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:17.897590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:17.957326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:17.961986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:17.962290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:17.962577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] PATH_INFO: `/v3/projects/b6c9127cf9a447bb958835891d053c52` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:17.967105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6c9127cf9a447bb958835891d053c52)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:17.969198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:18.019352 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6c9127cf9a447bb958835891d053c52', 'request_id': 'req-7c748762-cfea-44eb-9d34-7fcbfed18e4f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.026838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6c9127cf9a447bb958835891d053c52', 'request_id': 'req-7c748762-cfea-44eb-9d34-7fcbfed18e4f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.035686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6c9127cf9a447bb958835891d053c52', 'request_id': 'req-7c748762-cfea-44eb-9d34-7fcbfed18e4f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.050024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] The token cache is being invalidate because project b6c9127cf9a447bb958835891d053c52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:18.050321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.050981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.055976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1070/2119] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:17 2022] DELETE /identity/v3/projects/b6c9127cf9a447bb958835891d053c52 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:18.061254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97aa665-629b-4240-9ce9-d1a5be49a504 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.061551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97aa665-629b-4240-9ce9-d1a5be49a504 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.061837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97aa665-629b-4240-9ce9-d1a5be49a504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.082378 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a97aa665-629b-4240-9ce9-d1a5be49a504 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:18.084393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a97aa665-629b-4240-9ce9-d1a5be49a504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:18.113706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:18.117589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:18.150770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1050/2120] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:18.158383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef426323-8c1b-49f8-b13b-f7a435d9f2bb None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.158667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef426323-8c1b-49f8-b13b-f7a435d9f2bb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.158940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef426323-8c1b-49f8-b13b-f7a435d9f2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.166961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:18.182033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ef426323-8c1b-49f8-b13b-f7a435d9f2bb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:18.183248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ef426323-8c1b-49f8-b13b-f7a435d9f2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:18.189987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c748762-cfea-44eb-9d34-7fcbfed18e4f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:18.191657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1051/2121] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:18.195719 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd721a-1895-4c6b-9c05-c68d5b402338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:18.196363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd721a-1895-4c6b-9c05-c68d5b402338 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.196771 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef426323-8c1b-49f8-b13b-f7a435d9f2bb None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:18.196886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd721a-1895-4c6b-9c05-c68d5b402338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.197051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd721a-1895-4c6b-9c05-c68d5b402338 admin admin] PATH_INFO: `/v3/users/689e9243133b4401979a4a6d2f30c22d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.199262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd721a-1895-4c6b-9c05-c68d5b402338 admin admin] RBAC: Authorizing `identity:delete_user(user_id=689e9243133b4401979a4a6d2f30c22d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:18.201374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd721a-1895-4c6b-9c05-c68d5b402338 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:18.206179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef426323-8c1b-49f8-b13b-f7a435d9f2bb None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:18.211585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15d870-88c7-472e-948e-b0abb84c041c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:18.212510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15d870-88c7-472e-948e-b0abb84c041c admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.212817 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15d870-88c7-472e-948e-b0abb84c041c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.213093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb15d870-88c7-472e-948e-b0abb84c041c admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.213827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15d870-88c7-472e-948e-b0abb84c041c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:12:18.214125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15d870-88c7-472e-948e-b0abb84c041c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:18.215717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb15d870-88c7-472e-948e-b0abb84c041c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:18.230741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1052/2122] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:12:18.235247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15d870-88c7-472e-948e-b0abb84c041c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:18.248111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb15d870-88c7-472e-948e-b0abb84c041c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:18.254466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a184f-00ba-41e2-a9be-0c7da30a6a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:18.255311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a184f-00ba-41e2-a9be-0c7da30a6a97 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.255311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a184f-00ba-41e2-a9be-0c7da30a6a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.256223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a184f-00ba-41e2-a9be-0c7da30a6a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.256930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a184f-00ba-41e2-a9be-0c7da30a6a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:18.258899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072a184f-00ba-41e2-a9be-0c7da30a6a97 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:18.264475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f2cd721a-1895-4c6b-9c05-c68d5b402338 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '689e9243133b4401979a4a6d2f30c22d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.265179 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f2cd721a-1895-4c6b-9c05-c68d5b402338 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '689e9243133b4401979a4a6d2f30c22d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.265830 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f2cd721a-1895-4c6b-9c05-c68d5b402338 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '689e9243133b4401979a4a6d2f30c22d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.270425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:18.281353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f2cd721a-1895-4c6b-9c05-c68d5b402338 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '689e9243133b4401979a4a6d2f30c22d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.288894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f2cd721a-1895-4c6b-9c05-c68d5b402338 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '689e9243133b4401979a4a6d2f30c22d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.289856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1053/2123] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:18 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:18.294171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a184f-00ba-41e2-a9be-0c7da30a6a97 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:18.296590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-072a184f-00ba-41e2-a9be-0c7da30a6a97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:18.306857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1071/2124] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:18 2022] DELETE /identity/v3/users/689e9243133b4401979a4a6d2f30c22d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:18.311950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e226e0-b8bf-46de-b216-1d8daecd06bc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.312058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e226e0-b8bf-46de-b216-1d8daecd06bc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.312275 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e226e0-b8bf-46de-b216-1d8daecd06bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.332689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-05e226e0-b8bf-46de-b216-1d8daecd06bc None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:18.334341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-05e226e0-b8bf-46de-b216-1d8daecd06bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:18.372565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a184f-00ba-41e2-a9be-0c7da30a6a97 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:18.397957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1072/2125] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:18.407848 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e226e0-b8bf-46de-b216-1d8daecd06bc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:18.409313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-05e226e0-b8bf-46de-b216-1d8daecd06bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:18.439973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06456661-0123-401d-9f48-cde011f3f7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:18.440616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06456661-0123-401d-9f48-cde011f3f7e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.440894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06456661-0123-401d-9f48-cde011f3f7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.441195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06456661-0123-401d-9f48-cde011f3f7e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.441871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06456661-0123-401d-9f48-cde011f3f7e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:18.444826 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06456661-0123-401d-9f48-cde011f3f7e2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:18.469397 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1054/2126] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:18 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:18.469588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e226e0-b8bf-46de-b216-1d8daecd06bc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:18.470382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06456661-0123-401d-9f48-cde011f3f7e2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:18.470952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032df3fa-e38a-449b-b3c9-71af3476f878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:18.471581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032df3fa-e38a-449b-b3c9-71af3476f878 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.471903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032df3fa-e38a-449b-b3c9-71af3476f878 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.472228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032df3fa-e38a-449b-b3c9-71af3476f878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.480574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06456661-0123-401d-9f48-cde011f3f7e2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:18.483910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032df3fa-e38a-449b-b3c9-71af3476f878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:18.485412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032df3fa-e38a-449b-b3c9-71af3476f878 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:18.485530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb85ce-544b-4b50-9b56-8e3ad9c29ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:18.486008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb85ce-544b-4b50-9b56-8e3ad9c29ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.486288 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb85ce-544b-4b50-9b56-8e3ad9c29ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.486561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fb85ce-544b-4b50-9b56-8e3ad9c29ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.487206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb85ce-544b-4b50-9b56-8e3ad9c29ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:18.488789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fb85ce-544b-4b50-9b56-8e3ad9c29ab5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:18.495806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1073/2127] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:12:18.500021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1055/2128] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:12:18.504014 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032df3fa-e38a-449b-b3c9-71af3476f878 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:18.516252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032df3fa-e38a-449b-b3c9-71af3476f878 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:18.528589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791c822-6c78-4dd2-9567-d4e8a52a322a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:18.528589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791c822-6c78-4dd2-9567-d4e8a52a322a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.528589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791c822-6c78-4dd2-9567-d4e8a52a322a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.528990 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791c822-6c78-4dd2-9567-d4e8a52a322a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.529528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791c822-6c78-4dd2-9567-d4e8a52a322a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:18.531156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791c822-6c78-4dd2-9567-d4e8a52a322a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:18.545319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1074/2129] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:18 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:18.548005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb85ce-544b-4b50-9b56-8e3ad9c29ab5 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:18.559264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fb85ce-544b-4b50-9b56-8e3ad9c29ab5 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:18.565212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47374b6-de9a-497d-852b-7c99c1bffd9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:18.565889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47374b6-de9a-497d-852b-7c99c1bffd9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.566168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47374b6-de9a-497d-852b-7c99c1bffd9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.566443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47374b6-de9a-497d-852b-7c99c1bffd9d admin admin] PATH_INFO: `/v3/projects/aabd05caaf3a44dda3790d8d642cdc86/users/44a1dfda54db494bb7a2550b15c2ba51/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.582094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47374b6-de9a-497d-852b-7c99c1bffd9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=aabd05caaf3a44dda3790d8d642cdc86, user_id=44a1dfda54db494bb7a2550b15c2ba51, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:18.585302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47374b6-de9a-497d-852b-7c99c1bffd9d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:18.600718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1056/2130] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:18 2022] PUT /identity/v3/projects/aabd05caaf3a44dda3790d8d642cdc86/users/44a1dfda54db494bb7a2550b15c2ba51/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:18.611014 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b636a-4c7d-4b91-868a-897acc91bbb5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.611311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b636a-4c7d-4b91-868a-897acc91bbb5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.611599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b636a-4c7d-4b91-868a-897acc91bbb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.644058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c73b636a-4c7d-4b91-868a-897acc91bbb5 None None] MFA Rules not processed for user `44a1dfda54db494bb7a2550b15c2ba51`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:18.645386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c73b636a-4c7d-4b91-868a-897acc91bbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:18.699590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:18.725199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1075/2131] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:18.728371 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47374b6-de9a-497d-852b-7c99c1bffd9d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:18.774641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:18.828317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47374b6-de9a-497d-852b-7c99c1bffd9d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:18.833795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b072e-1b25-48fd-ab75-0fae975637ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:18.834785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b072e-1b25-48fd-ab75-0fae975637ab admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.835065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b072e-1b25-48fd-ab75-0fae975637ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.835334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b072e-1b25-48fd-ab75-0fae975637ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.836338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b072e-1b25-48fd-ab75-0fae975637ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:18.838291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b072e-1b25-48fd-ab75-0fae975637ab admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:18.852159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b636a-4c7d-4b91-868a-897acc91bbb5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:18.855260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c73b636a-4c7d-4b91-868a-897acc91bbb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:18.877910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b636a-4c7d-4b91-868a-897acc91bbb5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:18.888497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:18.889292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:18.889704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:18.889979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] PATH_INFO: `/v3/projects/a3ec7e60c54d4fa19f6e5e839e99e220` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:18.891257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3ec7e60c54d4fa19f6e5e839e99e220)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:18.893415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:18.897428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1057/2132] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:18 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:18.898329 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b072e-1b25-48fd-ab75-0fae975637ab admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:18.957509 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3ec7e60c54d4fa19f6e5e839e99e220', 'request_id': 'req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.966074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3ec7e60c54d4fa19f6e5e839e99e220', 'request_id': 'req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.981564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3ec7e60c54d4fa19f6e5e839e99e220', 'request_id': 'req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.983018 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b072e-1b25-48fd-ab75-0fae975637ab admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:18.997264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] The token cache is being invalidate because project a3ec7e60c54d4fa19f6e5e839e99e220 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:18.997592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:18.998394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:19.002787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1076/2133] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:18 2022] DELETE /identity/v3/projects/a3ec7e60c54d4fa19f6e5e839e99e220 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:19.003959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:19.005385 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:19.063764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2b582-43c9-4382-8bf1-87f73d8ba083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:19.064409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2b582-43c9-4382-8bf1-87f73d8ba083 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:19.064691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2b582-43c9-4382-8bf1-87f73d8ba083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:19.068867 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2b582-43c9-4382-8bf1-87f73d8ba083 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:19.068867 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2b582-43c9-4382-8bf1-87f73d8ba083 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:19.069431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2b582-43c9-4382-8bf1-87f73d8ba083 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:19.089902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b765ce6-36cd-45c8-9468-f9ae0ef95cb4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:19.093138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1058/2134] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:18 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:19.097325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2b582-43c9-4382-8bf1-87f73d8ba083 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:19.098597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:19.099263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:19.100069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:19.100189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-55b2b582-43c9-4382-8bf1-87f73d8ba083 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:19.100346 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] PATH_INFO: `/v3/users/50d668ac07a142bd846df7c521943b1d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:19.101524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d668ac07a142bd846df7c521943b1d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:19.103477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:19.113002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2b582-43c9-4382-8bf1-87f73d8ba083 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:19.119723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dfaf4-6033-4b30-bad2-eb3ddcf37adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:19.121194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dfaf4-6033-4b30-bad2-eb3ddcf37adc admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:19.121544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dfaf4-6033-4b30-bad2-eb3ddcf37adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:19.121887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dfaf4-6033-4b30-bad2-eb3ddcf37adc admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:19.122612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dfaf4-6033-4b30-bad2-eb3ddcf37adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:19.124291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dfaf4-6033-4b30-bad2-eb3ddcf37adc admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:19.139749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1059/2135] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:19.144163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dfaf4-6033-4b30-bad2-eb3ddcf37adc admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:19.156781 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dfaf4-6033-4b30-bad2-eb3ddcf37adc admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:19.162802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3837a-9b6a-4973-b137-a76596956666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:19.163854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3837a-9b6a-4973-b137-a76596956666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:19.163994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d668ac07a142bd846df7c521943b1d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:19.164490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3837a-9b6a-4973-b137-a76596956666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:19.164608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d668ac07a142bd846df7c521943b1d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:19.165160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3837a-9b6a-4973-b137-a76596956666 admin admin] PATH_INFO: `/v3/projects/02aa0791882e4667a89ba8a5e957e837/users/c595fe67a7784a73b83fa7ce76699b82/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:19.165272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d668ac07a142bd846df7c521943b1d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:19.179282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d668ac07a142bd846df7c521943b1d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:19.184234 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3837a-9b6a-4973-b137-a76596956666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02aa0791882e4667a89ba8a5e957e837, user_id=c595fe67a7784a73b83fa7ce76699b82, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:19.187002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3837a-9b6a-4973-b137-a76596956666 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:19.188425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d668ac07a142bd846df7c521943b1d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:19.191322 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:19.209444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1060/2136] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:19 2022] PUT /identity/v3/projects/02aa0791882e4667a89ba8a5e957e837/users/c595fe67a7784a73b83fa7ce76699b82/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:19.213713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1077/2137] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:19 2022] DELETE /identity/v3/users/50d668ac07a142bd846df7c521943b1d => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:19.214705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:19.216182 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:19.300265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5afec-d481-4189-97b3-66f6512b40e8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:19.307367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8516de-5933-4864-b8b5-af6653899369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:19.308286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8516de-5933-4864-b8b5-af6653899369 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:19.308802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8516de-5933-4864-b8b5-af6653899369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:19.309307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8516de-5933-4864-b8b5-af6653899369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:19.312345 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8516de-5933-4864-b8b5-af6653899369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:19.312565 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8516de-5933-4864-b8b5-af6653899369 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:19.327710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1078/2138] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:19.331808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3837a-9b6a-4973-b137-a76596956666 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:19.351137 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:19.444822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3837a-9b6a-4973-b137-a76596956666 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:19.450297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f057449-a35e-4a90-8fa0-107f7dbb8405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:19.451717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f057449-a35e-4a90-8fa0-107f7dbb8405 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:19.452153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f057449-a35e-4a90-8fa0-107f7dbb8405 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:19.455083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f057449-a35e-4a90-8fa0-107f7dbb8405 admin admin] PATH_INFO: `/v3/projects/02aa0791882e4667a89ba8a5e957e837/users/c595fe67a7784a73b83fa7ce76699b82/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:19.458316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f057449-a35e-4a90-8fa0-107f7dbb8405 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02aa0791882e4667a89ba8a5e957e837, user_id=c595fe67a7784a73b83fa7ce76699b82, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:19.465815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f057449-a35e-4a90-8fa0-107f7dbb8405 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:19.474934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1061/2139] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:19 2022] PUT /identity/v3/projects/02aa0791882e4667a89ba8a5e957e837/users/c595fe67a7784a73b83fa7ce76699b82/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:19.485878 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886538b-e7a9-4c85-a0bc-2b8999e4c904 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:19.486524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886538b-e7a9-4c85-a0bc-2b8999e4c904 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:19.487034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8886538b-e7a9-4c85-a0bc-2b8999e4c904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:19.521194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8886538b-e7a9-4c85-a0bc-2b8999e4c904 None None] MFA Rules not processed for user `c595fe67a7784a73b83fa7ce76699b82`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:19.523456 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8886538b-e7a9-4c85-a0bc-2b8999e4c904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:19.550400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f057449-a35e-4a90-8fa0-107f7dbb8405 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:19.552391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4f057449-a35e-4a90-8fa0-107f7dbb8405 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:19.617333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1079/2140] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:19 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:19.628666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886538b-e7a9-4c85-a0bc-2b8999e4c904 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:19.630335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8886538b-e7a9-4c85-a0bc-2b8999e4c904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:19.649886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f057449-a35e-4a90-8fa0-107f7dbb8405 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:19.654629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:19.655477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:19.656133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:19.656133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] PATH_INFO: `/v3/projects/b12b9023e05f4591bc945451b9457059` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:19.657336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b12b9023e05f4591bc945451b9457059)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:19.663982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:19.701160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8886538b-e7a9-4c85-a0bc-2b8999e4c904 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:19.706457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b45dd-2f62-4a0b-bde5-e96b4c3001e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:19.707542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b45dd-2f62-4a0b-bde5-e96b4c3001e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:19.707542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b45dd-2f62-4a0b-bde5-e96b4c3001e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:19.709412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b45dd-2f62-4a0b-bde5-e96b4c3001e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:19.719732 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b45dd-2f62-4a0b-bde5-e96b4c3001e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:19.724968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b45dd-2f62-4a0b-bde5-e96b4c3001e1 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:19.742040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1080/2141] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:19 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:12:19.747498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b12b9023e05f4591bc945451b9457059', 'request_id': 'req-0a5f809b-f227-414f-bb8c-b156cc9f642c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:19.766430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b12b9023e05f4591bc945451b9457059', 'request_id': 'req-0a5f809b-f227-414f-bb8c-b156cc9f642c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:19.782182 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b12b9023e05f4591bc945451b9457059', 'request_id': 'req-0a5f809b-f227-414f-bb8c-b156cc9f642c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:19.805256 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] The token cache is being invalidate because project b12b9023e05f4591bc945451b9457059 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:19.805256 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:19.806164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0a5f809b-f227-414f-bb8c-b156cc9f642c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:19.810994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1062/2142] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:19 2022] DELETE /identity/v3/projects/b12b9023e05f4591bc945451b9457059 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:19.814721 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ade57a-8b53-4f58-b0ad-03fa7ef7f211 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:19.815003 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ade57a-8b53-4f58-b0ad-03fa7ef7f211 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:19.815274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ade57a-8b53-4f58-b0ad-03fa7ef7f211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:19.843749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a8ade57a-8b53-4f58-b0ad-03fa7ef7f211 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:19.845512 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a8ade57a-8b53-4f58-b0ad-03fa7ef7f211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:19.864408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:19.955064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1081/2143] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:19.965147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2379c6-f366-4f01-87dd-78f68eb63285 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:19.965147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2379c6-f366-4f01-87dd-78f68eb63285 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:19.965457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2379c6-f366-4f01-87dd-78f68eb63285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:19.998249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ee2379c6-f366-4f01-87dd-78f68eb63285 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:19.999921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ee2379c6-f366-4f01-87dd-78f68eb63285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:20.013332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1063/2144] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:20.014363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ade57a-8b53-4f58-b0ad-03fa7ef7f211 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:20.029200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ade57a-8b53-4f58-b0ad-03fa7ef7f211 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:20.045591 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145eac6-e2b5-41ff-8aa9-fdb2095b8613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:20.046347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145eac6-e2b5-41ff-8aa9-fdb2095b8613 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:20.046750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145eac6-e2b5-41ff-8aa9-fdb2095b8613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:20.047129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9145eac6-e2b5-41ff-8aa9-fdb2095b8613 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:20.047972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145eac6-e2b5-41ff-8aa9-fdb2095b8613 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:12:20.048382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145eac6-e2b5-41ff-8aa9-fdb2095b8613 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:20.050426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9145eac6-e2b5-41ff-8aa9-fdb2095b8613 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:20.075000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1082/2145] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:12:20.084316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2379c6-f366-4f01-87dd-78f68eb63285 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:20.096383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2379c6-f366-4f01-87dd-78f68eb63285 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:20.104313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71500ffc-7eca-4b9f-8181-d3bfa60ff549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:20.105136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71500ffc-7eca-4b9f-8181-d3bfa60ff549 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:20.105610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71500ffc-7eca-4b9f-8181-d3bfa60ff549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:20.106013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71500ffc-7eca-4b9f-8181-d3bfa60ff549 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:20.106814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71500ffc-7eca-4b9f-8181-d3bfa60ff549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:20.112912 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71500ffc-7eca-4b9f-8181-d3bfa60ff549 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:20.157157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1064/2146] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:20 2022] POST /identity/v3/projects => generated 373 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:20.158703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145eac6-e2b5-41ff-8aa9-fdb2095b8613 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:20.174101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9145eac6-e2b5-41ff-8aa9-fdb2095b8613 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:20.257309 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715c069-e64b-499e-8473-bfe32270c4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:20.257935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715c069-e64b-499e-8473-bfe32270c4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:20.258215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715c069-e64b-499e-8473-bfe32270c4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:20.258486 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5715c069-e64b-499e-8473-bfe32270c4d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:20.259129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715c069-e64b-499e-8473-bfe32270c4d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:20.261125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5715c069-e64b-499e-8473-bfe32270c4d3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:20.286903 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1083/2147] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:20 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:20.291169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71500ffc-7eca-4b9f-8181-d3bfa60ff549 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:20.298477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:20.304541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71500ffc-7eca-4b9f-8181-d3bfa60ff549 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:20.311156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1e517-b55b-4e19-82e8-ed69f9d0cb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:20.311370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1e517-b55b-4e19-82e8-ed69f9d0cb1b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:20.311548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1e517-b55b-4e19-82e8-ed69f9d0cb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:20.312273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1e517-b55b-4e19-82e8-ed69f9d0cb1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:20.313304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1e517-b55b-4e19-82e8-ed69f9d0cb1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:20.315278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1e517-b55b-4e19-82e8-ed69f9d0cb1b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:20.333438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715c069-e64b-499e-8473-bfe32270c4d3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:20.334637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1065/2148] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:20.346659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5715c069-e64b-499e-8473-bfe32270c4d3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:20.352681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb69cf-3eaa-4351-a8bd-295f20f18b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:20.353636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb69cf-3eaa-4351-a8bd-295f20f18b80 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:20.353920 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb69cf-3eaa-4351-a8bd-295f20f18b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:20.354199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb69cf-3eaa-4351-a8bd-295f20f18b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:20.355057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb69cf-3eaa-4351-a8bd-295f20f18b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:20.357150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb69cf-3eaa-4351-a8bd-295f20f18b80 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:20.360254 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1e517-b55b-4e19-82e8-ed69f9d0cb1b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:20.362225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e8c1e517-b55b-4e19-82e8-ed69f9d0cb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:20.375445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1e517-b55b-4e19-82e8-ed69f9d0cb1b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:20.378507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1084/2149] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:20 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:20.383347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb69cf-3eaa-4351-a8bd-295f20f18b80 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:20.384494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5c351-df97-4f29-b68b-84f1182bba12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:20.385173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5c351-df97-4f29-b68b-84f1182bba12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:20.385459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5c351-df97-4f29-b68b-84f1182bba12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:20.385735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5c351-df97-4f29-b68b-84f1182bba12 admin admin] PATH_INFO: `/v3/users/44a1dfda54db494bb7a2550b15c2ba51` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:20.386925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5c351-df97-4f29-b68b-84f1182bba12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44a1dfda54db494bb7a2550b15c2ba51)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:20.388864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5c351-df97-4f29-b68b-84f1182bba12 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:20.405187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb69cf-3eaa-4351-a8bd-295f20f18b80 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:20.410907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1c77fa-277a-402b-b12c-57ea8edab5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:20.412387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c77fa-277a-402b-b12c-57ea8edab5ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:20.412699 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c77fa-277a-402b-b12c-57ea8edab5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:20.413001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1c77fa-277a-402b-b12c-57ea8edab5ee admin admin] PATH_INFO: `/v3/projects/d08945e124d14a05988b92fd9f236881/users/13e4bf53716c4714863cad6dace792dd/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:20.430583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c77fa-277a-402b-b12c-57ea8edab5ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d08945e124d14a05988b92fd9f236881, user_id=13e4bf53716c4714863cad6dace792dd, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:20.433244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1c77fa-277a-402b-b12c-57ea8edab5ee admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:20.446756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e2d5c351-df97-4f29-b68b-84f1182bba12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44a1dfda54db494bb7a2550b15c2ba51'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:20.447667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e2d5c351-df97-4f29-b68b-84f1182bba12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44a1dfda54db494bb7a2550b15c2ba51'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:20.453209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1085/2150] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:20 2022] PUT /identity/v3/projects/d08945e124d14a05988b92fd9f236881/users/13e4bf53716c4714863cad6dace792dd/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:20.454599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f939b-9622-4807-923f-c040321626c1 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:20.454883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f939b-9622-4807-923f-c040321626c1 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:20.455148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31f939b-9622-4807-923f-c040321626c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:20.461221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e2d5c351-df97-4f29-b68b-84f1182bba12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44a1dfda54db494bb7a2550b15c2ba51'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:20.462266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e2d5c351-df97-4f29-b68b-84f1182bba12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44a1dfda54db494bb7a2550b15c2ba51'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:20.477465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e31f939b-9622-4807-923f-c040321626c1 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:20.478612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e31f939b-9622-4807-923f-c040321626c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:20.481621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e2d5c351-df97-4f29-b68b-84f1182bba12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44a1dfda54db494bb7a2550b15c2ba51'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:20.499685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1066/2151] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:20 2022] DELETE /identity/v3/users/44a1dfda54db494bb7a2550b15c2ba51 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:20.505044 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f139d-371e-4671-b830-52bc102299ca None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:20.505893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f139d-371e-4671-b830-52bc102299ca None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:20.506176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465f139d-371e-4671-b830-52bc102299ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:20.529309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-465f139d-371e-4671-b830-52bc102299ca None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:20.530514 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-465f139d-371e-4671-b830-52bc102299ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:20.540718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:20.554645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1086/2152] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:20.565367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f939b-9622-4807-923f-c040321626c1 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:20.566764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e31f939b-9622-4807-923f-c040321626c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:20.602122 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1067/2153] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:20.608499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f139d-371e-4671-b830-52bc102299ca None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:20.610163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-465f139d-371e-4671-b830-52bc102299ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:20.637214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31f939b-9622-4807-923f-c040321626c1 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:20.675769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465f139d-371e-4671-b830-52bc102299ca None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:20.681380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5717538-c3d1-447b-bdad-83bb8e267837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:20.682566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5717538-c3d1-447b-bdad-83bb8e267837 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:20.682859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5717538-c3d1-447b-bdad-83bb8e267837 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:20.683130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5717538-c3d1-447b-bdad-83bb8e267837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:20.691423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8278d-4f0c-4535-a29b-cba59b49535b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:20.692595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8278d-4f0c-4535-a29b-cba59b49535b service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:20.692935 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8278d-4f0c-4535-a29b-cba59b49535b service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:20.693250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8278d-4f0c-4535-a29b-cba59b49535b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:20.694356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5717538-c3d1-447b-bdad-83bb8e267837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:20.695876 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5717538-c3d1-447b-bdad-83bb8e267837 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:20.702905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8278d-4f0c-4535-a29b-cba59b49535b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:20.704373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d8278d-4f0c-4535-a29b-cba59b49535b service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:20.713065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1068/2154] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:20.715016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1087/2155] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:12:21.471457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5717538-c3d1-447b-bdad-83bb8e267837 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:21.472997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e5717538-c3d1-447b-bdad-83bb8e267837 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:21.488196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5717538-c3d1-447b-bdad-83bb8e267837 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:21.515124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:21.516278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:21.516778 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:21.518650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] PATH_INFO: `/v3/projects/aabd05caaf3a44dda3790d8d642cdc86` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:21.522802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aabd05caaf3a44dda3790d8d642cdc86)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:21.525228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:21.533696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:21.625845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aabd05caaf3a44dda3790d8d642cdc86', 'request_id': 'req-d446f2d9-d652-46a5-a9ea-8732614fc794'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:21.637887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aabd05caaf3a44dda3790d8d642cdc86', 'request_id': 'req-d446f2d9-d652-46a5-a9ea-8732614fc794'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:21.649358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aabd05caaf3a44dda3790d8d642cdc86', 'request_id': 'req-d446f2d9-d652-46a5-a9ea-8732614fc794'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:21.667284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] The token cache is being invalidate because project aabd05caaf3a44dda3790d8d642cdc86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:21.667626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:21.668813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:21.670412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1069/2156] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:21 2022] DELETE /identity/v3/projects/aabd05caaf3a44dda3790d8d642cdc86 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:21.674469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8278d-4f0c-4535-a29b-cba59b49535b service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:21.685388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8278d-4f0c-4535-a29b-cba59b49535b service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:21.751886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc941a-29ac-4cf0-81f1-fda0c1ef2eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:21.752518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc941a-29ac-4cf0-81f1-fda0c1ef2eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:21.752987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc941a-29ac-4cf0-81f1-fda0c1ef2eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:21.753290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc941a-29ac-4cf0-81f1-fda0c1ef2eb2 admin admin] PATH_INFO: `/v3/users/c595fe67a7784a73b83fa7ce76699b82` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:21.754455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc941a-29ac-4cf0-81f1-fda0c1ef2eb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c595fe67a7784a73b83fa7ce76699b82)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:21.757301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc941a-29ac-4cf0-81f1-fda0c1ef2eb2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:21.813389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-eefc941a-29ac-4cf0-81f1-fda0c1ef2eb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c595fe67a7784a73b83fa7ce76699b82'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:21.813389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-eefc941a-29ac-4cf0-81f1-fda0c1ef2eb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c595fe67a7784a73b83fa7ce76699b82'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:21.813389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-eefc941a-29ac-4cf0-81f1-fda0c1ef2eb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c595fe67a7784a73b83fa7ce76699b82'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:21.824639 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:21.826808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-eefc941a-29ac-4cf0-81f1-fda0c1ef2eb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c595fe67a7784a73b83fa7ce76699b82'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:21.836639 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-eefc941a-29ac-4cf0-81f1-fda0c1ef2eb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c595fe67a7784a73b83fa7ce76699b82'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:21.850524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1088/2157] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:21 2022] DELETE /identity/v3/users/c595fe67a7784a73b83fa7ce76699b82 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:22.220169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:22.222237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:22.248043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446f2d9-d652-46a5-a9ea-8732614fc794 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:22.329525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:22.330621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:22.330906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:22.331184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] PATH_INFO: `/v3/projects/02aa0791882e4667a89ba8a5e957e837` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:22.332346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02aa0791882e4667a89ba8a5e957e837)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:22.335213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:22.359983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69264fc-50b3-4072-9f51-ddade1a7d7bb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:22.360273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69264fc-50b3-4072-9f51-ddade1a7d7bb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:22.360547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69264fc-50b3-4072-9f51-ddade1a7d7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:22.404270 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f69264fc-50b3-4072-9f51-ddade1a7d7bb None None] MFA Rules not processed for user `13e4bf53716c4714863cad6dace792dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:22.405740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f69264fc-50b3-4072-9f51-ddade1a7d7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:22.411835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02aa0791882e4667a89ba8a5e957e837', 'request_id': 'req-38a86d26-c04b-4875-9eea-6cfe467b6098'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:22.422923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02aa0791882e4667a89ba8a5e957e837', 'request_id': 'req-38a86d26-c04b-4875-9eea-6cfe467b6098'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:22.432002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02aa0791882e4667a89ba8a5e957e837', 'request_id': 'req-38a86d26-c04b-4875-9eea-6cfe467b6098'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:22.464696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] The token cache is being invalidate because project 02aa0791882e4667a89ba8a5e957e837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:22.464696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:22.465655 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38a86d26-c04b-4875-9eea-6cfe467b6098 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:22.471526 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1070/2158] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:22 2022] DELETE /identity/v3/projects/02aa0791882e4667a89ba8a5e957e837 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:22.477782 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d65be9-a504-4c1e-adfe-25921d9d2ae5 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:22.481030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d65be9-a504-4c1e-adfe-25921d9d2ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:22.481577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d65be9-a504-4c1e-adfe-25921d9d2ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:22.539624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a8d65be9-a504-4c1e-adfe-25921d9d2ae5 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:22.541214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a8d65be9-a504-4c1e-adfe-25921d9d2ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:22.544965 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1089/2159] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:22.549377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69264fc-50b3-4072-9f51-ddade1a7d7bb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:22.550967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f69264fc-50b3-4072-9f51-ddade1a7d7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:22.570732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:22.661952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69264fc-50b3-4072-9f51-ddade1a7d7bb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:22.667895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf267d-e8e8-4d3e-b261-a317a038e06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:22.668506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf267d-e8e8-4d3e-b261-a317a038e06b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:22.668780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf267d-e8e8-4d3e-b261-a317a038e06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:22.669045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bf267d-e8e8-4d3e-b261-a317a038e06b admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:22.669708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf267d-e8e8-4d3e-b261-a317a038e06b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:22.672124 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bf267d-e8e8-4d3e-b261-a317a038e06b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:22.679460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1071/2160] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:22.688686 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f938ab-a7a9-49de-9301-ac9adbb141dc None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:22.689219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f938ab-a7a9-49de-9301-ac9adbb141dc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:22.689499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f938ab-a7a9-49de-9301-ac9adbb141dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:22.716237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-85f938ab-a7a9-49de-9301-ac9adbb141dc None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:22.719714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-85f938ab-a7a9-49de-9301-ac9adbb141dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:22.721744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1090/2161] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:22 2022] POST /identity/v3/projects => generated 373 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:22.725942 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf267d-e8e8-4d3e-b261-a317a038e06b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:22.827629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:22.860066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1072/2162] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:22.864078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f938ab-a7a9-49de-9301-ac9adbb141dc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:22.866989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bf267d-e8e8-4d3e-b261-a317a038e06b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:22.872248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f334901-c6cc-4c9d-95b8-b4524b499b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:22.872906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334901-c6cc-4c9d-95b8-b4524b499b54 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:22.873538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334901-c6cc-4c9d-95b8-b4524b499b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:22.873538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f334901-c6cc-4c9d-95b8-b4524b499b54 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:22.874997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f334901-c6cc-4c9d-95b8-b4524b499b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:22.882608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f334901-c6cc-4c9d-95b8-b4524b499b54 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:22.883074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f938ab-a7a9-49de-9301-ac9adbb141dc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:22.896652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1264c-a7e7-42a7-b12a-6db379d12ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:22.897298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1264c-a7e7-42a7-b12a-6db379d12ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:22.897502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1264c-a7e7-42a7-b12a-6db379d12ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:22.897774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1264c-a7e7-42a7-b12a-6db379d12ce2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:22.898427 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1264c-a7e7-42a7-b12a-6db379d12ce2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:12:22.898732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1264c-a7e7-42a7-b12a-6db379d12ce2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:22.900343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1264c-a7e7-42a7-b12a-6db379d12ce2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:22.930068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1091/2163] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:22 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:22.930268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1073/2164] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:12:22.931596 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1264c-a7e7-42a7-b12a-6db379d12ce2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:22.934813 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f334901-c6cc-4c9d-95b8-b4524b499b54 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:22.951940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1264c-a7e7-42a7-b12a-6db379d12ce2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:22.953973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f334901-c6cc-4c9d-95b8-b4524b499b54 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:22.959860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e602b-ef9a-4338-b2bd-a19e373f1fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:22.960477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e602b-ef9a-4338-b2bd-a19e373f1fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:22.960762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e602b-ef9a-4338-b2bd-a19e373f1fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:22.961237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e602b-ef9a-4338-b2bd-a19e373f1fa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:22.961893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e602b-ef9a-4338-b2bd-a19e373f1fa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:22.962356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df330b1-b926-48cf-a64e-b8f4b3a31e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:22.963045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df330b1-b926-48cf-a64e-b8f4b3a31e9a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:22.963393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df330b1-b926-48cf-a64e-b8f4b3a31e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:22.963736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df330b1-b926-48cf-a64e-b8f4b3a31e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:22.964452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df330b1-b926-48cf-a64e-b8f4b3a31e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:22.964655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e602b-ef9a-4338-b2bd-a19e373f1fa6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:22.967371 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df330b1-b926-48cf-a64e-b8f4b3a31e9a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:22.988441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1074/2165] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:22 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:22.990803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df330b1-b926-48cf-a64e-b8f4b3a31e9a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:23.017714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1092/2166] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:22 2022] POST /identity/v3/projects => generated 383 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:23.019024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e602b-ef9a-4338-b2bd-a19e373f1fa6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:23.027683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df330b1-b926-48cf-a64e-b8f4b3a31e9a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:23.120372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ebd67-f761-4cfc-bdd5-538a02775246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:23.121047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ebd67-f761-4cfc-bdd5-538a02775246 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:23.121354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ebd67-f761-4cfc-bdd5-538a02775246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:23.121633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ebd67-f761-4cfc-bdd5-538a02775246 admin admin] PATH_INFO: `/v3/projects/cc5ce9a880b24221aa780b393c751bd9/users/3321328f852243d39eed306776104ce4/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:23.140287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ebd67-f761-4cfc-bdd5-538a02775246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5ce9a880b24221aa780b393c751bd9, user_id=3321328f852243d39eed306776104ce4, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:23.141463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e602b-ef9a-4338-b2bd-a19e373f1fa6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:23.142988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ebd67-f761-4cfc-bdd5-538a02775246 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:23.148402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347295e3-fbdd-40ee-883e-9e3a439b509b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:23.149533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347295e3-fbdd-40ee-883e-9e3a439b509b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:23.150142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347295e3-fbdd-40ee-883e-9e3a439b509b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:23.150754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347295e3-fbdd-40ee-883e-9e3a439b509b admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:23.152300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347295e3-fbdd-40ee-883e-9e3a439b509b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:23.154770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347295e3-fbdd-40ee-883e-9e3a439b509b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:23.167868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1075/2167] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:22 2022] PUT /identity/v3/projects/cc5ce9a880b24221aa780b393c751bd9/users/3321328f852243d39eed306776104ce4/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:23.171478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ebd67-f761-4cfc-bdd5-538a02775246 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:23.194453 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1093/2168] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:23 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:23.198916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347295e3-fbdd-40ee-883e-9e3a439b509b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:23.211757 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347295e3-fbdd-40ee-883e-9e3a439b509b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:23.223603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:23.274889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ebd67-f761-4cfc-bdd5-538a02775246 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:23.280350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fd373-19a9-44ca-924d-1ad781db424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:23.281198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd373-19a9-44ca-924d-1ad781db424a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:23.281550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd373-19a9-44ca-924d-1ad781db424a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:23.281909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd373-19a9-44ca-924d-1ad781db424a admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:23.282754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342fd373-19a9-44ca-924d-1ad781db424a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:23.284668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342fd373-19a9-44ca-924d-1ad781db424a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:23.294351 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10714-7206-47ee-bd1e-11cb743e2c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:23.295203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10714-7206-47ee-bd1e-11cb743e2c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:23.295555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10714-7206-47ee-bd1e-11cb743e2c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:23.295993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d10714-7206-47ee-bd1e-11cb743e2c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:23.296937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d10714-7206-47ee-bd1e-11cb743e2c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:23.297060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1076/2169] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:12:23.298499 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d10714-7206-47ee-bd1e-11cb743e2c63 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:23.301422 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fd373-19a9-44ca-924d-1ad781db424a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:23.312698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342fd373-19a9-44ca-924d-1ad781db424a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:23.313201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1094/2170] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:23.316863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10714-7206-47ee-bd1e-11cb743e2c63 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:23.323553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91e340-5c6e-41fa-b51a-e6dd93217384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:23.324904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91e340-5c6e-41fa-b51a-e6dd93217384 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:23.324904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91e340-5c6e-41fa-b51a-e6dd93217384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:23.325147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f91e340-5c6e-41fa-b51a-e6dd93217384 admin admin] PATH_INFO: `/v3/projects/cc5ce9a880b24221aa780b393c751bd9/users/3321328f852243d39eed306776104ce4/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:23.328141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f91e340-5c6e-41fa-b51a-e6dd93217384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc5ce9a880b24221aa780b393c751bd9, user_id=3321328f852243d39eed306776104ce4, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:23.328590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d10714-7206-47ee-bd1e-11cb743e2c63 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:23.333003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f91e340-5c6e-41fa-b51a-e6dd93217384 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:23.334871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d9cf2-35c8-466b-9b3d-db397d95bada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:23.335587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d9cf2-35c8-466b-9b3d-db397d95bada admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:23.335943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d9cf2-35c8-466b-9b3d-db397d95bada admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:23.336294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d9cf2-35c8-466b-9b3d-db397d95bada admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:23.337343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d9cf2-35c8-466b-9b3d-db397d95bada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:23.340164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d9cf2-35c8-466b-9b3d-db397d95bada admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:23.353525 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1095/2172] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:23.357192 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1077/2172] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:23 2022] PUT /identity/v3/projects/cc5ce9a880b24221aa780b393c751bd9/users/3321328f852243d39eed306776104ce4/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:23.366409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d9cf2-35c8-466b-9b3d-db397d95bada admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:23.380107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d9cf2-35c8-466b-9b3d-db397d95bada admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:23.445307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:23.497857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b346f4ec-6d94-4714-bec2-873373c9c188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:23.499031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346f4ec-6d94-4714-bec2-873373c9c188 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:23.499535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346f4ec-6d94-4714-bec2-873373c9c188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:23.500028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b346f4ec-6d94-4714-bec2-873373c9c188 admin admin] PATH_INFO: `/v3/projects/edde81efbf2a4ec8b294287f6dd5360e/users/8d4957f5e40e4527a5a5f06779c3ebd8/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:23.534400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346f4ec-6d94-4714-bec2-873373c9c188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edde81efbf2a4ec8b294287f6dd5360e, user_id=8d4957f5e40e4527a5a5f06779c3ebd8, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:23.541311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b346f4ec-6d94-4714-bec2-873373c9c188 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:23.554849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91e340-5c6e-41fa-b51a-e6dd93217384 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:23.557570 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4f91e340-5c6e-41fa-b51a-e6dd93217384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:23.586648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1096/2173] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:23 2022] PUT /identity/v3/projects/edde81efbf2a4ec8b294287f6dd5360e/users/8d4957f5e40e4527a5a5f06779c3ebd8/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:23.589417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f91e340-5c6e-41fa-b51a-e6dd93217384 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:23.599458 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce8583-37d8-4f88-96bd-cfc3d5d6204b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:23.600015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce8583-37d8-4f88-96bd-cfc3d5d6204b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:23.600527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ce8583-37d8-4f88-96bd-cfc3d5d6204b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:23.648210 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b9ce8583-37d8-4f88-96bd-cfc3d5d6204b None None] MFA Rules not processed for user `8d4957f5e40e4527a5a5f06779c3ebd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:23.649635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b9ce8583-37d8-4f88-96bd-cfc3d5d6204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:23.692524 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:23.694304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:23.694884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:23.695326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] PATH_INFO: `/v3/users/1c9c1aa9349f43e9b56ea4248c3c7f36` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:23.696474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c9c1aa9349f43e9b56ea4248c3c7f36)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:23.698987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:23.753340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1097/2174] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:23.763608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c9c1aa9349f43e9b56ea4248c3c7f36'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:23.764297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c9c1aa9349f43e9b56ea4248c3c7f36'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:23.768276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce8583-37d8-4f88-96bd-cfc3d5d6204b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:23.769931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b9ce8583-37d8-4f88-96bd-cfc3d5d6204b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:23.780322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c9c1aa9349f43e9b56ea4248c3c7f36'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:23.781160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c9c1aa9349f43e9b56ea4248c3c7f36'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:23.801229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c9c1aa9349f43e9b56ea4248c3c7f36'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:23.840828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1078/2175] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:23 2022] DELETE /identity/v3/users/1c9c1aa9349f43e9b56ea4248c3c7f36 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:23.860428 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ce8583-37d8-4f88-96bd-cfc3d5d6204b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:23.946424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4a168-3c76-4049-a843-487ab6390d7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:23.947156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4a168-3c76-4049-a843-487ab6390d7e service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:23.947511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4a168-3c76-4049-a843-487ab6390d7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:23.947863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e4a168-3c76-4049-a843-487ab6390d7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:23.949934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-25e4a168-3c76-4049-a843-487ab6390d7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:23.972949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:24.020886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4a168-3c76-4049-a843-487ab6390d7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.023367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e4a168-3c76-4049-a843-487ab6390d7e service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.033727 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1098/2176] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:24.054916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:24.056339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:24.089349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:24.149774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5768bf-2f0a-48d6-9f46-d77386b8e148 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:24.157396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1f2c2-3658-4b14-8876-6241e3b51d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:24.158322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1f2c2-3658-4b14-8876-6241e3b51d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.158745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1f2c2-3658-4b14-8876-6241e3b51d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.159049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1f2c2-3658-4b14-8876-6241e3b51d73 admin admin] PATH_INFO: `/v3/users/8d4957f5e40e4527a5a5f06779c3ebd8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.163457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1f2c2-3658-4b14-8876-6241e3b51d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d4957f5e40e4527a5a5f06779c3ebd8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.168359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1f2c2-3658-4b14-8876-6241e3b51d73 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.199688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4a168-3c76-4049-a843-487ab6390d7e service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:24.202044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-25e4a168-3c76-4049-a843-487ab6390d7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:24.231154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e4a168-3c76-4049-a843-487ab6390d7e service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:24.236439 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:24.237519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.238018 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.238548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] PATH_INFO: `/v3/projects/2a52e44d03f74727863b596c606ee35d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.240553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a52e44d03f74727863b596c606ee35d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.242717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.273765 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-65b1f2c2-3658-4b14-8876-6241e3b51d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d4957f5e40e4527a5a5f06779c3ebd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:24.274837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-65b1f2c2-3658-4b14-8876-6241e3b51d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d4957f5e40e4527a5a5f06779c3ebd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:24.284340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-65b1f2c2-3658-4b14-8876-6241e3b51d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d4957f5e40e4527a5a5f06779c3ebd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:24.285045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-65b1f2c2-3658-4b14-8876-6241e3b51d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d4957f5e40e4527a5a5f06779c3ebd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:24.301258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-65b1f2c2-3658-4b14-8876-6241e3b51d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d4957f5e40e4527a5a5f06779c3ebd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:24.311820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a52e44d03f74727863b596c606ee35d', 'request_id': 'req-8e4bbb05-a425-44f1-9e02-7138f5da9a39'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:24.321449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1079/2177] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:24 2022] DELETE /identity/v3/users/8d4957f5e40e4527a5a5f06779c3ebd8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:24.321655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a52e44d03f74727863b596c606ee35d', 'request_id': 'req-8e4bbb05-a425-44f1-9e02-7138f5da9a39'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:24.321769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9e429-2636-4a40-b1c5-ab6755cb31cd None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.321769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9e429-2636-4a40-b1c5-ab6755cb31cd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.321769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9e429-2636-4a40-b1c5-ab6755cb31cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.328042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a52e44d03f74727863b596c606ee35d', 'request_id': 'req-8e4bbb05-a425-44f1-9e02-7138f5da9a39'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:24.345887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] The token cache is being invalidate because project 2a52e44d03f74727863b596c606ee35d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:24.346173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:24.347020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e4bbb05-a425-44f1-9e02-7138f5da9a39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:24.348709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1099/2178] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:24 2022] DELETE /identity/v3/projects/2a52e44d03f74727863b596c606ee35d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:24.355178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915482-b37a-4d5f-ad36-0af7a17716e6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.355463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915482-b37a-4d5f-ad36-0af7a17716e6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.355734 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4915482-b37a-4d5f-ad36-0af7a17716e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.373340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8cf9e429-2636-4a40-b1c5-ab6755cb31cd None None] MFA Rules not processed for user `3321328f852243d39eed306776104ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:24.374661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8cf9e429-2636-4a40-b1c5-ab6755cb31cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:24.376815 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e4915482-b37a-4d5f-ad36-0af7a17716e6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:24.378143 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e4915482-b37a-4d5f-ad36-0af7a17716e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:24.463558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1100/2179] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:24.463558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0523e-45f4-4a0f-8889-718907f39147 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.463558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0523e-45f4-4a0f-8889-718907f39147 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.463558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0523e-45f4-4a0f-8889-718907f39147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.469554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1080/2180] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:24.470791 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9e429-2636-4a40-b1c5-ab6755cb31cd None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:24.472171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8cf9e429-2636-4a40-b1c5-ab6755cb31cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:24.493368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2ad0523e-45f4-4a0f-8889-718907f39147 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:24.495211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2ad0523e-45f4-4a0f-8889-718907f39147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:24.506660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1101/2181] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:24.510199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:24.510726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0523e-45f4-4a0f-8889-718907f39147 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:24.512287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2ad0523e-45f4-4a0f-8889-718907f39147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:24.540784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf9e429-2636-4a40-b1c5-ab6755cb31cd None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:24.544476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9142c468-878a-40f5-a404-505903826279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:24.545075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c468-878a-40f5-a404-505903826279 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.545384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c468-878a-40f5-a404-505903826279 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.545652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9142c468-878a-40f5-a404-505903826279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.555267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142c468-878a-40f5-a404-505903826279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.556696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9142c468-878a-40f5-a404-505903826279 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.559959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:24.567174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1081/2182] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:24.568610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad476a1-d199-461d-bdf9-6e12910f51eb None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.568938 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad476a1-d199-461d-bdf9-6e12910f51eb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.569245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad476a1-d199-461d-bdf9-6e12910f51eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.590443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0523e-45f4-4a0f-8889-718907f39147 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:24.591257 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0ad476a1-d199-461d-bdf9-6e12910f51eb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:24.592387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0ad476a1-d199-461d-bdf9-6e12910f51eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:24.594166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7ab0c-2743-4fd7-bb8f-cc7d26e5b03c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:24.594776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7ab0c-2743-4fd7-bb8f-cc7d26e5b03c service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.595053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7ab0c-2743-4fd7-bb8f-cc7d26e5b03c service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.595381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b7ab0c-2743-4fd7-bb8f-cc7d26e5b03c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.597041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-45b7ab0c-2743-4fd7-bb8f-cc7d26e5b03c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:24.604365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1082/2183] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:24.608323 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad476a1-d199-461d-bdf9-6e12910f51eb None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:24.617962 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad476a1-d199-461d-bdf9-6e12910f51eb None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:24.622911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544a010-3cff-495d-96a0-a28e6968591f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:24.623504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a010-3cff-495d-96a0-a28e6968591f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.623776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a010-3cff-495d-96a0-a28e6968591f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.624043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0544a010-3cff-495d-96a0-a28e6968591f admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.624674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a010-3cff-495d-96a0-a28e6968591f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:12:24.624975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a010-3cff-495d-96a0-a28e6968591f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.626598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0544a010-3cff-495d-96a0-a28e6968591f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.643301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1083/2184] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:12:24.644204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544a010-3cff-495d-96a0-a28e6968591f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:24.656406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0544a010-3cff-495d-96a0-a28e6968591f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:24.662849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a678e-0db0-4aa9-93c5-496adae83033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:24.663968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a678e-0db0-4aa9-93c5-496adae83033 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.664258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a678e-0db0-4aa9-93c5-496adae83033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.664635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a678e-0db0-4aa9-93c5-496adae83033 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.665200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a678e-0db0-4aa9-93c5-496adae83033 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.668442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a678e-0db0-4aa9-93c5-496adae83033 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.680672 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7ab0c-2743-4fd7-bb8f-cc7d26e5b03c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.682602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b7ab0c-2743-4fd7-bb8f-cc7d26e5b03c service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.695161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1102/2185] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:24.703312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1084/2186] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:24 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:24.707948 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7ab0c-2743-4fd7-bb8f-cc7d26e5b03c service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:24.720001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b7ab0c-2743-4fd7-bb8f-cc7d26e5b03c service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:24.801673 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f6e32-c8f2-440c-bee5-a1c94f6b0478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:24.802731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f6e32-c8f2-440c-bee5-a1c94f6b0478 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.803016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f6e32-c8f2-440c-bee5-a1c94f6b0478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.803293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f6e32-c8f2-440c-bee5-a1c94f6b0478 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.803936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f6e32-c8f2-440c-bee5-a1c94f6b0478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.806068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f6e32-c8f2-440c-bee5-a1c94f6b0478 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.830330 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1103/2187] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:24 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:24.835355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a678e-0db0-4aa9-93c5-496adae83033 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:24.849930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a678e-0db0-4aa9-93c5-496adae83033 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:24.856791 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c2ef1-0bdd-4705-a4c8-513f9d7d34db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:24.857437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c2ef1-0bdd-4705-a4c8-513f9d7d34db admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.857772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c2ef1-0bdd-4705-a4c8-513f9d7d34db admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.858046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4c2ef1-0bdd-4705-a4c8-513f9d7d34db admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.858722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c2ef1-0bdd-4705-a4c8-513f9d7d34db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.860905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4c2ef1-0bdd-4705-a4c8-513f9d7d34db admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.872150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1085/2188] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:24.875851 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f6e32-c8f2-440c-bee5-a1c94f6b0478 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:24.886595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f6e32-c8f2-440c-bee5-a1c94f6b0478 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:24.892942 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee1737-8437-4dde-97b9-76f773468639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:24.893582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee1737-8437-4dde-97b9-76f773468639 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.893928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee1737-8437-4dde-97b9-76f773468639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.894135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ee1737-8437-4dde-97b9-76f773468639 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.894968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee1737-8437-4dde-97b9-76f773468639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.897324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ee1737-8437-4dde-97b9-76f773468639 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.906355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c2ef1-0bdd-4705-a4c8-513f9d7d34db admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:24.908403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1f4c2ef1-0bdd-4705-a4c8-513f9d7d34db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:24.910467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1104/2189] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:24.914916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee1737-8437-4dde-97b9-76f773468639 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:24.929707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4c2ef1-0bdd-4705-a4c8-513f9d7d34db admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:24.934663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:24.935346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.935542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.935827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] PATH_INFO: `/v3/projects/edde81efbf2a4ec8b294287f6dd5360e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.936975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edde81efbf2a4ec8b294287f6dd5360e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.938884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.942214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ee1737-8437-4dde-97b9-76f773468639 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:24.947879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065c307-039c-4770-b219-3c6daa5b8f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:24.949066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065c307-039c-4770-b219-3c6daa5b8f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.949377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065c307-039c-4770-b219-3c6daa5b8f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.949765 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065c307-039c-4770-b219-3c6daa5b8f89 admin admin] PATH_INFO: `/v3/projects/650863eae15e4733b50e22c7595d7742/users/62e294148e14459a96190fab931d4898/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:24.969057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065c307-039c-4770-b219-3c6daa5b8f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=650863eae15e4733b50e22c7595d7742, user_id=62e294148e14459a96190fab931d4898, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:24.972838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065c307-039c-4770-b219-3c6daa5b8f89 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:24.987914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1105/2190] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:24 2022] PUT /identity/v3/projects/650863eae15e4733b50e22c7595d7742/users/62e294148e14459a96190fab931d4898/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:24.993123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f10c1b-03fb-4e36-95ec-a7b1e89a17c8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:24.993505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f10c1b-03fb-4e36-95ec-a7b1e89a17c8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:24.993809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f10c1b-03fb-4e36-95ec-a7b1e89a17c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.011926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edde81efbf2a4ec8b294287f6dd5360e', 'request_id': 'req-4a0f651e-e64a-4bbf-8adf-2aca85de6610'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:25.016875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-00f10c1b-03fb-4e36-95ec-a7b1e89a17c8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:25.018335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-00f10c1b-03fb-4e36-95ec-a7b1e89a17c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:25.030743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edde81efbf2a4ec8b294287f6dd5360e', 'request_id': 'req-4a0f651e-e64a-4bbf-8adf-2aca85de6610'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:25.039866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edde81efbf2a4ec8b294287f6dd5360e', 'request_id': 'req-4a0f651e-e64a-4bbf-8adf-2aca85de6610'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:25.070629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:25.071443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] The token cache is being invalidate because project edde81efbf2a4ec8b294287f6dd5360e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:25.071798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:25.072762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4a0f651e-e64a-4bbf-8adf-2aca85de6610 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:25.081298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1086/2191] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:24 2022] DELETE /identity/v3/projects/edde81efbf2a4ec8b294287f6dd5360e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:25.081298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34087-32a3-45dd-b81c-0196b01fe0d8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:25.081298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34087-32a3-45dd-b81c-0196b01fe0d8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:25.081298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd34087-32a3-45dd-b81c-0196b01fe0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.104039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ecd34087-32a3-45dd-b81c-0196b01fe0d8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:25.105470 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ecd34087-32a3-45dd-b81c-0196b01fe0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:25.106756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1106/2192] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:25.117218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f10c1b-03fb-4e36-95ec-a7b1e89a17c8 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:25.118796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-00f10c1b-03fb-4e36-95ec-a7b1e89a17c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:25.124802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1087/2193] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:25.133378 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e8da9-3206-4362-988c-32ff23b1808e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:25.133701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e8da9-3206-4362-988c-32ff23b1808e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:25.133941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e8da9-3206-4362-988c-32ff23b1808e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.153741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0f1e8da9-3206-4362-988c-32ff23b1808e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:25.154945 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0f1e8da9-3206-4362-988c-32ff23b1808e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:25.163353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1088/2194] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:25.167488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e8da9-3206-4362-988c-32ff23b1808e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:25.176388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e8da9-3206-4362-988c-32ff23b1808e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:25.180316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f10c1b-03fb-4e36-95ec-a7b1e89a17c8 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:25.181711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f994237-b77c-4681-87da-10a96cf0b00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:25.182335 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f994237-b77c-4681-87da-10a96cf0b00f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:25.182618 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f994237-b77c-4681-87da-10a96cf0b00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:25.182889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f994237-b77c-4681-87da-10a96cf0b00f admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.183786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f994237-b77c-4681-87da-10a96cf0b00f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:12:25.184089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f994237-b77c-4681-87da-10a96cf0b00f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:25.185750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f994237-b77c-4681-87da-10a96cf0b00f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:25.186047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee32c1d-ebb1-4bc4-b673-f916df95c9db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:25.186891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee32c1d-ebb1-4bc4-b673-f916df95c9db service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:25.187439 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee32c1d-ebb1-4bc4-b673-f916df95c9db service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:25.187986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee32c1d-ebb1-4bc4-b673-f916df95c9db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.198668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1089/2195] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:12:25.202868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee32c1d-ebb1-4bc4-b673-f916df95c9db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:25.204292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee32c1d-ebb1-4bc4-b673-f916df95c9db service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:25.210104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f994237-b77c-4681-87da-10a96cf0b00f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:25.216974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1107/2196] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:25.223045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f994237-b77c-4681-87da-10a96cf0b00f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:25.229584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddab48d-46e5-413e-a63d-60a254df3c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:25.230326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddab48d-46e5-413e-a63d-60a254df3c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:25.230607 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddab48d-46e5-413e-a63d-60a254df3c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:25.230897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddab48d-46e5-413e-a63d-60a254df3c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.231640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddab48d-46e5-413e-a63d-60a254df3c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:25.233767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddab48d-46e5-413e-a63d-60a254df3c01 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:25.269139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1090/2197] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:25 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:25.273552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee32c1d-ebb1-4bc4-b673-f916df95c9db service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:25.283310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee32c1d-ebb1-4bc4-b673-f916df95c9db service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:25.343343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c24065-c1c4-4d73-a6b6-c170e291a8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:25.343959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c24065-c1c4-4d73-a6b6-c170e291a8cb admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:25.344241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c24065-c1c4-4d73-a6b6-c170e291a8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:25.344559 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c24065-c1c4-4d73-a6b6-c170e291a8cb admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.345251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c24065-c1c4-4d73-a6b6-c170e291a8cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:25.347248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c24065-c1c4-4d73-a6b6-c170e291a8cb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:25.370290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1108/2198] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:25 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:25.374676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddab48d-46e5-413e-a63d-60a254df3c01 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:25.386355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddab48d-46e5-413e-a63d-60a254df3c01 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:25.391941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b545b-97d4-4bde-b541-1e2a3e0c65ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:25.392815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b545b-97d4-4bde-b541-1e2a3e0c65ab admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:25.393120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b545b-97d4-4bde-b541-1e2a3e0c65ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:25.393399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b545b-97d4-4bde-b541-1e2a3e0c65ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.394333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b545b-97d4-4bde-b541-1e2a3e0c65ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:25.395724 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b545b-97d4-4bde-b541-1e2a3e0c65ab admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:25.417289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1091/2199] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:25.417469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c24065-c1c4-4d73-a6b6-c170e291a8cb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:25.426555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c24065-c1c4-4d73-a6b6-c170e291a8cb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:25.432887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11b09b-b42f-4124-bdff-91d16c3694af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:25.433747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11b09b-b42f-4124-bdff-91d16c3694af admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:25.434140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11b09b-b42f-4124-bdff-91d16c3694af admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:25.434315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11b09b-b42f-4124-bdff-91d16c3694af admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.434995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11b09b-b42f-4124-bdff-91d16c3694af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:25.436593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb11b09b-b42f-4124-bdff-91d16c3694af admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:25.448858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1109/2200] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:12:25.452992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b545b-97d4-4bde-b541-1e2a3e0c65ab admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:25.458527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:25.477766 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b545b-97d4-4bde-b541-1e2a3e0c65ab admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:25.484197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef010545-e189-49e1-a744-869b60673f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:25.485562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef010545-e189-49e1-a744-869b60673f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:25.486041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef010545-e189-49e1-a744-869b60673f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:25.486777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef010545-e189-49e1-a744-869b60673f30 admin admin] PATH_INFO: `/v3/projects/d3df93047f3c41e58dce763a98bdef11/users/400acfcfa7b84c16914480b53d34b4cc/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.505357 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef010545-e189-49e1-a744-869b60673f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3df93047f3c41e58dce763a98bdef11, user_id=400acfcfa7b84c16914480b53d34b4cc, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:25.508385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef010545-e189-49e1-a744-869b60673f30 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:25.522115 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1092/2201] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:25 2022] PUT /identity/v3/projects/d3df93047f3c41e58dce763a98bdef11/users/400acfcfa7b84c16914480b53d34b4cc/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:12:25.532271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d833750-0725-464c-8cf6-483e816f4b35 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:25.532939 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d833750-0725-464c-8cf6-483e816f4b35 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:25.533288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d833750-0725-464c-8cf6-483e816f4b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.564392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8d833750-0725-464c-8cf6-483e816f4b35 None None] MFA Rules not processed for user `400acfcfa7b84c16914480b53d34b4cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:25.566578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8d833750-0725-464c-8cf6-483e816f4b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:25.630198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:25.641584 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1110/2202] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:25.652760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef010545-e189-49e1-a744-869b60673f30 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:25.654534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ef010545-e189-49e1-a744-869b60673f30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:25.735933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef010545-e189-49e1-a744-869b60673f30 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:25.740517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791344c5-e796-4adb-a2fd-8544c286ca76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:25.741193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791344c5-e796-4adb-a2fd-8544c286ca76 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:25.741470 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791344c5-e796-4adb-a2fd-8544c286ca76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:25.741745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791344c5-e796-4adb-a2fd-8544c286ca76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:25.750248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791344c5-e796-4adb-a2fd-8544c286ca76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:25.751677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791344c5-e796-4adb-a2fd-8544c286ca76 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:25.761365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1093/2203] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:28.322977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d833750-0725-464c-8cf6-483e816f4b35 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:28.324690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8d833750-0725-464c-8cf6-483e816f4b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:28.348839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d833750-0725-464c-8cf6-483e816f4b35 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:28.354867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7859bc-2d4f-4c47-8397-4365076ea4d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:28.355410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7859bc-2d4f-4c47-8397-4365076ea4d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:28.355695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7859bc-2d4f-4c47-8397-4365076ea4d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:28.355968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7859bc-2d4f-4c47-8397-4365076ea4d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:28.360007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1c7859bc-2d4f-4c47-8397-4365076ea4d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:28.481429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7859bc-2d4f-4c47-8397-4365076ea4d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:28.482994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7859bc-2d4f-4c47-8397-4365076ea4d9 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:28.494886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1111/2204] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:29.747749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccb039-7930-4d95-8c01-42463dec4289 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:29.748454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccb039-7930-4d95-8c01-42463dec4289 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:29.748737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeccb039-7930-4d95-8c01-42463dec4289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:29.788922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-aeccb039-7930-4d95-8c01-42463dec4289 None None] MFA Rules not processed for user `62e294148e14459a96190fab931d4898`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:29.791161 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-aeccb039-7930-4d95-8c01-42463dec4289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:29.897261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1094/2205] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:29.913043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7859bc-2d4f-4c47-8397-4365076ea4d9 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:29.914786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1c7859bc-2d4f-4c47-8397-4365076ea4d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:30.045387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7859bc-2d4f-4c47-8397-4365076ea4d9 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:30.052427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4fa5d-b17c-4411-896b-a055ebe07e99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:30.053369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4fa5d-b17c-4411-896b-a055ebe07e99 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:30.053822 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4fa5d-b17c-4411-896b-a055ebe07e99 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:30.054261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d4fa5d-b17c-4411-896b-a055ebe07e99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:30.071404 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4fa5d-b17c-4411-896b-a055ebe07e99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:30.073298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d4fa5d-b17c-4411-896b-a055ebe07e99 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:30.086552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1112/2206] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:31.967157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c1c9c-f9e5-4711-8512-387bbd0ec54c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:31.967626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c1c9c-f9e5-4711-8512-387bbd0ec54c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:31.967741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1c1c9c-f9e5-4711-8512-387bbd0ec54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:31.986870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8a1c1c9c-f9e5-4711-8512-387bbd0ec54c None None] MFA Rules not processed for user `62e294148e14459a96190fab931d4898`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:31.988123 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8a1c1c9c-f9e5-4711-8512-387bbd0ec54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:31.996344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1095/2207] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:40.081667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=66 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:41.981964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=70,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:46.131727 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4fa5d-b17c-4411-896b-a055ebe07e99 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:46.133239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e0d4fa5d-b17c-4411-896b-a055ebe07e99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:46.200265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d4fa5d-b17c-4411-896b-a055ebe07e99 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:46.204872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:46.205702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:46.205982 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:46.206256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] PATH_INFO: `/v3/users/6fe64cafb24648f9b69aa32c6a75fdaa` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:46.207416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe64cafb24648f9b69aa32c6a75fdaa)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:46.209498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:46.260036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe64cafb24648f9b69aa32c6a75fdaa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:46.260927 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe64cafb24648f9b69aa32c6a75fdaa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:46.261573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe64cafb24648f9b69aa32c6a75fdaa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:46.274270 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe64cafb24648f9b69aa32c6a75fdaa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:46.281589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe64cafb24648f9b69aa32c6a75fdaa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:46.297449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1113/2208] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:12:46 2022] DELETE /identity/v3/users/6fe64cafb24648f9b69aa32c6a75fdaa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:46.302965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe443c-9d37-4d5c-8473-d573ef3be18e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:46.303500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe443c-9d37-4d5c-8473-d573ef3be18e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:46.303500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fe443c-9d37-4d5c-8473-d573ef3be18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:46.322880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-30fe443c-9d37-4d5c-8473-d573ef3be18e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:46.324082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-30fe443c-9d37-4d5c-8473-d573ef3be18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:46.389239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1096/2209] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:46.418210 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:46.419613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:46.487303 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb08fb8-64e6-4f35-9462-e8820bd1f1fb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:46.492873 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91858c46-12c7-4f68-b962-2f7dfb90adf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:46.493718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91858c46-12c7-4f68-b962-2f7dfb90adf4 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:46.494214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91858c46-12c7-4f68-b962-2f7dfb90adf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:46.494451 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91858c46-12c7-4f68-b962-2f7dfb90adf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:46.516243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91858c46-12c7-4f68-b962-2f7dfb90adf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:46.516243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91858c46-12c7-4f68-b962-2f7dfb90adf4 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:46.523294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1114/2210] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:12:47.033566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe443c-9d37-4d5c-8473-d573ef3be18e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:47.035143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-30fe443c-9d37-4d5c-8473-d573ef3be18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:47.049048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fe443c-9d37-4d5c-8473-d573ef3be18e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:47.055016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:47.055745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:47.056267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:47.056765 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] PATH_INFO: `/v3/projects/3136faaea7794dd9bb86ca401fdee8bb` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:47.060361 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3136faaea7794dd9bb86ca401fdee8bb)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:47.062543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:47.120129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3136faaea7794dd9bb86ca401fdee8bb', 'request_id': 'req-19519e3b-2ee6-4208-b497-d30d216f0ddc'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:47.135837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3136faaea7794dd9bb86ca401fdee8bb', 'request_id': 'req-19519e3b-2ee6-4208-b497-d30d216f0ddc'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:47.144031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3136faaea7794dd9bb86ca401fdee8bb', 'request_id': 'req-19519e3b-2ee6-4208-b497-d30d216f0ddc'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:47.159102 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] The token cache is being invalidate because project 3136faaea7794dd9bb86ca401fdee8bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:12:47.159824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:47.161883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-19519e3b-2ee6-4208-b497-d30d216f0ddc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:12:47.163455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1097/2211] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:12:47 2022] DELETE /identity/v3/projects/3136faaea7794dd9bb86ca401fdee8bb => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:47.168518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2dfbc-e536-40b4-b2ce-3366ee347547 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:47.169028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2dfbc-e536-40b4-b2ce-3366ee347547 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:47.169142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf2dfbc-e536-40b4-b2ce-3366ee347547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:47.195738 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9cf2dfbc-e536-40b4-b2ce-3366ee347547 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:47.197502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9cf2dfbc-e536-40b4-b2ce-3366ee347547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:47.216992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:47.264287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1115/2212] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:47.273180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858cc75-bc89-4e9c-8290-d81d1563e16c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:47.273503 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858cc75-bc89-4e9c-8290-d81d1563e16c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:47.273775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7858cc75-bc89-4e9c-8290-d81d1563e16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:47.293613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7858cc75-bc89-4e9c-8290-d81d1563e16c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:47.294818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7858cc75-bc89-4e9c-8290-d81d1563e16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:47.303929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1098/2213] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:12:47.308206 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2dfbc-e536-40b4-b2ce-3366ee347547 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:47.319031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf2dfbc-e536-40b4-b2ce-3366ee347547 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:47.323997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2db78-1bf1-4ed6-be92-3ed25fcad1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:47.324728 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2db78-1bf1-4ed6-be92-3ed25fcad1ba admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:47.325133 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2db78-1bf1-4ed6-be92-3ed25fcad1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:47.325511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2db78-1bf1-4ed6-be92-3ed25fcad1ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:47.326359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2db78-1bf1-4ed6-be92-3ed25fcad1ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:12:47.326913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2db78-1bf1-4ed6-be92-3ed25fcad1ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:47.328894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2db78-1bf1-4ed6-be92-3ed25fcad1ba admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:47.343404 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1116/2214] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:12:47.347556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858cc75-bc89-4e9c-8290-d81d1563e16c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:47.357714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7858cc75-bc89-4e9c-8290-d81d1563e16c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:47.362733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f92a1b-e119-4907-a005-43281b8b147a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:47.363234 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f92a1b-e119-4907-a005-43281b8b147a admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:47.363516 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f92a1b-e119-4907-a005-43281b8b147a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:47.363787 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f92a1b-e119-4907-a005-43281b8b147a admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:47.364446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f92a1b-e119-4907-a005-43281b8b147a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:47.366412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f92a1b-e119-4907-a005-43281b8b147a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:47.400824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1099/2215] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:12:47 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:12:47.404922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2db78-1bf1-4ed6-be92-3ed25fcad1ba admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:47.416636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2db78-1bf1-4ed6-be92-3ed25fcad1ba admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:47.492049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127700bb-48ab-4eb6-bf44-3a106bd49f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:47.492674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127700bb-48ab-4eb6-bf44-3a106bd49f58 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:47.492955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127700bb-48ab-4eb6-bf44-3a106bd49f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:47.493331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127700bb-48ab-4eb6-bf44-3a106bd49f58 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:47.493913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127700bb-48ab-4eb6-bf44-3a106bd49f58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:47.495869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127700bb-48ab-4eb6-bf44-3a106bd49f58 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:47.522140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1117/2216] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:12:47 2022] POST /identity/v3/users => generated 283 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:12:47.523387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f92a1b-e119-4907-a005-43281b8b147a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:47.535129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f92a1b-e119-4907-a005-43281b8b147a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:47.540462 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c55529-7b31-4b89-8d93-81cccb4b4d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:47.541141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c55529-7b31-4b89-8d93-81cccb4b4d09 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:47.541434 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c55529-7b31-4b89-8d93-81cccb4b4d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:47.541707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c55529-7b31-4b89-8d93-81cccb4b4d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:47.542460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c55529-7b31-4b89-8d93-81cccb4b4d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:47.543982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c55529-7b31-4b89-8d93-81cccb4b4d09 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:47.558722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1100/2217] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:12:47.562568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127700bb-48ab-4eb6-bf44-3a106bd49f58 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:47.573411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127700bb-48ab-4eb6-bf44-3a106bd49f58 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:47.577987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0b6e3-a21a-4fe8-9326-173bdfaf0c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:47.578579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0b6e3-a21a-4fe8-9326-173bdfaf0c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:47.578853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0b6e3-a21a-4fe8-9326-173bdfaf0c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:47.579120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0b6e3-a21a-4fe8-9326-173bdfaf0c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:47.579752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0b6e3-a21a-4fe8-9326-173bdfaf0c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:47.581354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0b6e3-a21a-4fe8-9326-173bdfaf0c1b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:47.591909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1118/2218] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:12:47.595933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c55529-7b31-4b89-8d93-81cccb4b4d09 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:47.606788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c55529-7b31-4b89-8d93-81cccb4b4d09 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:47.611407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71660c-af4a-4351-b021-ba35a2585b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:47.612044 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71660c-af4a-4351-b021-ba35a2585b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:47.612354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71660c-af4a-4351-b021-ba35a2585b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:47.612631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d71660c-af4a-4351-b021-ba35a2585b8e admin admin] PATH_INFO: `/v3/projects/13b34455f657497ea9cef0305699e724/users/9c54c3844bc0421fbca966af7195843d/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:47.628130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71660c-af4a-4351-b021-ba35a2585b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=13b34455f657497ea9cef0305699e724, user_id=9c54c3844bc0421fbca966af7195843d, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:47.631349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d71660c-af4a-4351-b021-ba35a2585b8e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:47.644021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1101/2219] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:12:47 2022] PUT /identity/v3/projects/13b34455f657497ea9cef0305699e724/users/9c54c3844bc0421fbca966af7195843d/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:12:47.655277 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e7c06-f441-4784-be1d-0be62ea09274 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:47.655558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e7c06-f441-4784-be1d-0be62ea09274 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:47.655829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e7c06-f441-4784-be1d-0be62ea09274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:47.683260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f74e7c06-f441-4784-be1d-0be62ea09274 None None] MFA Rules not processed for user `9c54c3844bc0421fbca966af7195843d`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:12:47.684498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f74e7c06-f441-4784-be1d-0be62ea09274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:47.749379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1119/2220] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:12:47 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:12:47.762167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71660c-af4a-4351-b021-ba35a2585b8e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:12:47.764073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4d71660c-af4a-4351-b021-ba35a2585b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:12:47.790344 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:47.837934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d71660c-af4a-4351-b021-ba35a2585b8e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:12:47.843475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946a961-2d66-4607-8873-323370f8689e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:12:47.844090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946a961-2d66-4607-8873-323370f8689e service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:12:47.844574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946a961-2d66-4607-8873-323370f8689e service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:12:47.844721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946a961-2d66-4607-8873-323370f8689e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:12:47.854721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946a961-2d66-4607-8873-323370f8689e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:12:47.856941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946a961-2d66-4607-8873-323370f8689e service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:12:47.886710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1102/2221] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:12:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:12:57.745527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:12:57.880921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=17 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:00.762052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e7c06-f441-4784-be1d-0be62ea09274 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:00.763746 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f74e7c06-f441-4784-be1d-0be62ea09274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:00.900301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e7c06-f441-4784-be1d-0be62ea09274 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:00.908036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:00.910683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:00.915759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:00.916561 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] PATH_INFO: `/v3/users/400acfcfa7b84c16914480b53d34b4cc` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:00.918073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] RBAC: Authorizing `identity:delete_user(user_id=400acfcfa7b84c16914480b53d34b4cc)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:00.921020 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:01.004056 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '400acfcfa7b84c16914480b53d34b4cc'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.005365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '400acfcfa7b84c16914480b53d34b4cc'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.006133 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '400acfcfa7b84c16914480b53d34b4cc'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.029090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '400acfcfa7b84c16914480b53d34b4cc'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.048275 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '400acfcfa7b84c16914480b53d34b4cc'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.093340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1120/2222] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:13:00 2022] DELETE /identity/v3/users/400acfcfa7b84c16914480b53d34b4cc => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:13:01.093558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed4283-42e6-4120-8fbb-edbee6557d57 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:01.093558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed4283-42e6-4120-8fbb-edbee6557d57 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:01.093558 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ed4283-42e6-4120-8fbb-edbee6557d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:01.119168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-65ed4283-42e6-4120-8fbb-edbee6557d57 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:01.121286 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-65ed4283-42e6-4120-8fbb-edbee6557d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:01.208160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1103/2223] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:13:01.223795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:01.225496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:01.332054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6650420-3d02-4308-bc88-896c550dc81d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:01.337804 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f13a8-65bb-4ab6-882a-97888b407433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:01.338347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f13a8-65bb-4ab6-882a-97888b407433 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:01.338622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f13a8-65bb-4ab6-882a-97888b407433 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:01.338892 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0f13a8-65bb-4ab6-882a-97888b407433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:01.350885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f13a8-65bb-4ab6-882a-97888b407433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:01.352949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0f13a8-65bb-4ab6-882a-97888b407433 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:01.366467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1121/2224] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:13:01.652845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed4283-42e6-4120-8fbb-edbee6557d57 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:01.656565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-65ed4283-42e6-4120-8fbb-edbee6557d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:01.683852 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f13a8-65bb-4ab6-882a-97888b407433 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:01.685591 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9a0f13a8-65bb-4ab6-882a-97888b407433 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:01.685966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ed4283-42e6-4120-8fbb-edbee6557d57 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:01.691168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:01.692149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:01.692423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:01.692688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] PATH_INFO: `/v3/projects/d3df93047f3c41e58dce763a98bdef11` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:01.694035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3df93047f3c41e58dce763a98bdef11)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:01.695944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:01.700148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0f13a8-65bb-4ab6-882a-97888b407433 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:01.704807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95686d7b-baff-485f-8694-7493f0d7df3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:01.705464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95686d7b-baff-485f-8694-7493f0d7df3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:01.705748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95686d7b-baff-485f-8694-7493f0d7df3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:01.706025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95686d7b-baff-485f-8694-7493f0d7df3a admin admin] PATH_INFO: `/v3/users/13e4bf53716c4714863cad6dace792dd` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:01.708351 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95686d7b-baff-485f-8694-7493f0d7df3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=13e4bf53716c4714863cad6dace792dd)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:01.710331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95686d7b-baff-485f-8694-7493f0d7df3a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:01.726723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:01.749550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3df93047f3c41e58dce763a98bdef11', 'request_id': 'req-b6af1334-adef-4348-ab19-09e91553d3ef'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.758249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3df93047f3c41e58dce763a98bdef11', 'request_id': 'req-b6af1334-adef-4348-ab19-09e91553d3ef'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.764916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3df93047f3c41e58dce763a98bdef11', 'request_id': 'req-b6af1334-adef-4348-ab19-09e91553d3ef'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.765894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-95686d7b-baff-485f-8694-7493f0d7df3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13e4bf53716c4714863cad6dace792dd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.766607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-95686d7b-baff-485f-8694-7493f0d7df3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13e4bf53716c4714863cad6dace792dd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.767309 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-95686d7b-baff-485f-8694-7493f0d7df3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13e4bf53716c4714863cad6dace792dd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.776579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] The token cache is being invalidate because project d3df93047f3c41e58dce763a98bdef11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:13:01.776868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.777519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b6af1334-adef-4348-ab19-09e91553d3ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.778992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1104/2225] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:13:01 2022] DELETE /identity/v3/projects/d3df93047f3c41e58dce763a98bdef11 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:01.781723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-95686d7b-baff-485f-8694-7493f0d7df3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13e4bf53716c4714863cad6dace792dd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.783653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b34cc-ccbb-4835-88f9-adf2c73c60db None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:01.783940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b34cc-ccbb-4835-88f9-adf2c73c60db None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:01.784215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9b34cc-ccbb-4835-88f9-adf2c73c60db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:01.789460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-95686d7b-baff-485f-8694-7493f0d7df3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13e4bf53716c4714863cad6dace792dd'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:01.806490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1122/2226] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:13:01 2022] DELETE /identity/v3/users/13e4bf53716c4714863cad6dace792dd => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:13:01.809325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-cf9b34cc-ccbb-4835-88f9-adf2c73c60db None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:01.810589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cf9b34cc-ccbb-4835-88f9-adf2c73c60db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:01.879621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1105/2227] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:13:01.949128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc13cb-79e5-44ce-82e6-bf0d8d87d633 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:01.949475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc13cb-79e5-44ce-82e6-bf0d8d87d633 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:01.949755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc13cb-79e5-44ce-82e6-bf0d8d87d633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:01.970227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-18fc13cb-79e5-44ce-82e6-bf0d8d87d633 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:01.971515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-18fc13cb-79e5-44ce-82e6-bf0d8d87d633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:01.980795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1123/2228] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:01.984800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b34cc-ccbb-4835-88f9-adf2c73c60db None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:01.996247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9b34cc-ccbb-4835-88f9-adf2c73c60db None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:02.001837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc2469-fca7-47d4-a2c1-bd26661f0659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:02.002669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc2469-fca7-47d4-a2c1-bd26661f0659 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.003420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc2469-fca7-47d4-a2c1-bd26661f0659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.003420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fc2469-fca7-47d4-a2c1-bd26661f0659 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.004255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc2469-fca7-47d4-a2c1-bd26661f0659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:13:02.004365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc13cb-79e5-44ce-82e6-bf0d8d87d633 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:02.004715 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc2469-fca7-47d4-a2c1-bd26661f0659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:02.005785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-18fc13cb-79e5-44ce-82e6-bf0d8d87d633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:02.006841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fc2469-fca7-47d4-a2c1-bd26661f0659 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:02.020606 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fc13cb-79e5-44ce-82e6-bf0d8d87d633 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:02.021616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1106/2229] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:13:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:13:02.026514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:02.026699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc2469-fca7-47d4-a2c1-bd26661f0659 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:02.027157 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.027431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.027702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] PATH_INFO: `/v3/projects/d08945e124d14a05988b92fd9f236881` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.029150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d08945e124d14a05988b92fd9f236881)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:02.031041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:02.037408 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fc2469-fca7-47d4-a2c1-bd26661f0659 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:02.042386 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa7d49-416e-4610-b19a-ebedcdfa0510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:02.042931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa7d49-416e-4610-b19a-ebedcdfa0510 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.043167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa7d49-416e-4610-b19a-ebedcdfa0510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.043437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa7d49-416e-4610-b19a-ebedcdfa0510 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.044083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa7d49-416e-4610-b19a-ebedcdfa0510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:02.045987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fa7d49-416e-4610-b19a-ebedcdfa0510 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:02.057189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:02.079926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1107/2230] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:13:02 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:13:02.084633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa7d49-416e-4610-b19a-ebedcdfa0510 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:02.090204 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd08945e124d14a05988b92fd9f236881', 'request_id': 'req-5cf0563c-f5e3-4b53-9469-50c15fe75359'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.096383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd08945e124d14a05988b92fd9f236881', 'request_id': 'req-5cf0563c-f5e3-4b53-9469-50c15fe75359'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.103402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd08945e124d14a05988b92fd9f236881', 'request_id': 'req-5cf0563c-f5e3-4b53-9469-50c15fe75359'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.118975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] The token cache is being invalidate because project d08945e124d14a05988b92fd9f236881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:13:02.119279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.120131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.122528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1124/2231] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:13:02 2022] DELETE /identity/v3/projects/d08945e124d14a05988b92fd9f236881 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:02.125493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:02.127081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:02.157530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa7d49-416e-4610-b19a-ebedcdfa0510 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:02.163122 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892af3e-65ea-4f7b-9721-860883291f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:02.165786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892af3e-65ea-4f7b-9721-860883291f9b admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.166132 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892af3e-65ea-4f7b-9721-860883291f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.166406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5892af3e-65ea-4f7b-9721-860883291f9b admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.167059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892af3e-65ea-4f7b-9721-860883291f9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:02.169068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5892af3e-65ea-4f7b-9721-860883291f9b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:02.192853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1108/2232] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:13:02 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:13:02.196952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892af3e-65ea-4f7b-9721-860883291f9b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:02.198636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5892af3e-65ea-4f7b-9721-860883291f9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:02.212150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0563c-f5e3-4b53-9469-50c15fe75359 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:02.216625 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5892af3e-65ea-4f7b-9721-860883291f9b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:02.216841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b149fc-b28e-4897-b0d3-c3fbcf31fd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:02.217573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b149fc-b28e-4897-b0d3-c3fbcf31fd2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.217853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b149fc-b28e-4897-b0d3-c3fbcf31fd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.218124 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b149fc-b28e-4897-b0d3-c3fbcf31fd2d admin admin] PATH_INFO: `/v3/users/3321328f852243d39eed306776104ce4` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.219260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b149fc-b28e-4897-b0d3-c3fbcf31fd2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3321328f852243d39eed306776104ce4)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:02.221219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b149fc-b28e-4897-b0d3-c3fbcf31fd2d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:02.224920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3fb9f-c73b-4817-8237-1d23ced335fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:02.225585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3fb9f-c73b-4817-8237-1d23ced335fc admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.225861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3fb9f-c73b-4817-8237-1d23ced335fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.226130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3fb9f-c73b-4817-8237-1d23ced335fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.226779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3fb9f-c73b-4817-8237-1d23ced335fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:02.228380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3fb9f-c73b-4817-8237-1d23ced335fc admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:02.239438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:02.241398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1109/2233] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:13:02.245469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3fb9f-c73b-4817-8237-1d23ced335fc admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:02.257328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3fb9f-c73b-4817-8237-1d23ced335fc admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:02.263035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0388ad-0d45-4801-b6c0-73540e408357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:02.263856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0388ad-0d45-4801-b6c0-73540e408357 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.264150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0388ad-0d45-4801-b6c0-73540e408357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.264427 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0388ad-0d45-4801-b6c0-73540e408357 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.265072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0388ad-0d45-4801-b6c0-73540e408357 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:02.266724 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0388ad-0d45-4801-b6c0-73540e408357 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:02.278800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1110/2234] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:13:02.281294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17b149fc-b28e-4897-b0d3-c3fbcf31fd2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3321328f852243d39eed306776104ce4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.282007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17b149fc-b28e-4897-b0d3-c3fbcf31fd2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3321328f852243d39eed306776104ce4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.282672 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17b149fc-b28e-4897-b0d3-c3fbcf31fd2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3321328f852243d39eed306776104ce4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.283077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0388ad-0d45-4801-b6c0-73540e408357 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:02.285194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5d0388ad-0d45-4801-b6c0-73540e408357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:02.296211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17b149fc-b28e-4897-b0d3-c3fbcf31fd2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3321328f852243d39eed306776104ce4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.298803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0388ad-0d45-4801-b6c0-73540e408357 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:02.303866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17b149fc-b28e-4897-b0d3-c3fbcf31fd2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3321328f852243d39eed306776104ce4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.304478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65900ac-3773-465e-85b7-3b56d7c7c83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:02.305162 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65900ac-3773-465e-85b7-3b56d7c7c83e admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.305441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65900ac-3773-465e-85b7-3b56d7c7c83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.305706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65900ac-3773-465e-85b7-3b56d7c7c83e admin admin] PATH_INFO: `/v3/projects/756d38786c48457194521a27dee60af2/users/064ac808853f4133a3e4f3ed26c785ca/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.318321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1125/2235] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:13:02 2022] DELETE /identity/v3/users/3321328f852243d39eed306776104ce4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:13:02.323534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65900ac-3773-465e-85b7-3b56d7c7c83e admin admin] RBAC: Authorizing `identity:create_grant(project_id=756d38786c48457194521a27dee60af2, user_id=064ac808853f4133a3e4f3ed26c785ca, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:02.326874 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65900ac-3773-465e-85b7-3b56d7c7c83e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:02.343176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1111/2236] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:13:02 2022] PUT /identity/v3/projects/756d38786c48457194521a27dee60af2/users/064ac808853f4133a3e4f3ed26c785ca/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:02.354077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b997a1-9469-484a-814f-b02a33881f1d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.354571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b997a1-9469-484a-814f-b02a33881f1d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.355000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b997a1-9469-484a-814f-b02a33881f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.390063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-92b997a1-9469-484a-814f-b02a33881f1d None None] MFA Rules not processed for user `064ac808853f4133a3e4f3ed26c785ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:02.391308 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-92b997a1-9469-484a-814f-b02a33881f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:02.465718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1126/2237] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:02.578230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65900ac-3773-465e-85b7-3b56d7c7c83e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:02.580169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a65900ac-3773-465e-85b7-3b56d7c7c83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:02.680949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65900ac-3773-465e-85b7-3b56d7c7c83e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:02.682664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b997a1-9469-484a-814f-b02a33881f1d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:02.684228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-92b997a1-9469-484a-814f-b02a33881f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:02.686497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:02.687194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.687656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.687997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] PATH_INFO: `/v3/projects/cc5ce9a880b24221aa780b393c751bd9` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.689389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc5ce9a880b24221aa780b393c751bd9)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:02.692278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:02.740931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:02.760138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc5ce9a880b24221aa780b393c751bd9', 'request_id': 'req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.772476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc5ce9a880b24221aa780b393c751bd9', 'request_id': 'req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.785683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc5ce9a880b24221aa780b393c751bd9', 'request_id': 'req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.817357 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] The token cache is being invalidate because project cc5ce9a880b24221aa780b393c751bd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:13:02.818004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.819513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f231fbdc-b4e0-4c52-810e-1c1bc7ceabc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:02.821739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1112/2238] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:13:02 2022] DELETE /identity/v3/projects/cc5ce9a880b24221aa780b393c751bd9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:02.822679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b997a1-9469-484a-814f-b02a33881f1d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:02.826416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80521866-354a-48a8-8ae8-5e1d8924333b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.826708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80521866-354a-48a8-8ae8-5e1d8924333b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.827051 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80521866-354a-48a8-8ae8-5e1d8924333b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.849586 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-80521866-354a-48a8-8ae8-5e1d8924333b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:02.850921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-80521866-354a-48a8-8ae8-5e1d8924333b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:02.873568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:02.894540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392863b-04f8-492d-aab5-6a74003c6997 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:02.895466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392863b-04f8-492d-aab5-6a74003c6997 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.895868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392863b-04f8-492d-aab5-6a74003c6997 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.896250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d392863b-04f8-492d-aab5-6a74003c6997 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.898101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d392863b-04f8-492d-aab5-6a74003c6997 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:02.917936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1113/2239] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:02.926403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafbbce-d6ad-47b4-aa0e-52f162201893 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.926652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafbbce-d6ad-47b4-aa0e-52f162201893 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.926903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafbbce-d6ad-47b4-aa0e-52f162201893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.948958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-fdafbbce-d6ad-47b4-aa0e-52f162201893 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:02.950192 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fdafbbce-d6ad-47b4-aa0e-52f162201893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:02.959925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1114/2240] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:13:02.962341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392863b-04f8-492d-aab5-6a74003c6997 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:02.964131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d392863b-04f8-492d-aab5-6a74003c6997 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:02.964261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafbbce-d6ad-47b4-aa0e-52f162201893 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:02.973621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1127/2241] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:13:02.973870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafbbce-d6ad-47b4-aa0e-52f162201893 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:02.978629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8d419-987e-42b2-818b-fbdc42a3fb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:02.979317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d419-987e-42b2-818b-fbdc42a3fb63 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:02.979637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d419-987e-42b2-818b-fbdc42a3fb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:02.979862 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd8d419-987e-42b2-818b-fbdc42a3fb63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:02.980544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8d419-987e-42b2-818b-fbdc42a3fb63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:13:02.980804 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8d419-987e-42b2-818b-fbdc42a3fb63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:02.982431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd8d419-987e-42b2-818b-fbdc42a3fb63 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:02.999480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1115/2242] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:13:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:13:03.000897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392863b-04f8-492d-aab5-6a74003c6997 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.002383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d392863b-04f8-492d-aab5-6a74003c6997 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:03.004031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8d419-987e-42b2-818b-fbdc42a3fb63 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.014116 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd8d419-987e-42b2-818b-fbdc42a3fb63 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.016331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d392863b-04f8-492d-aab5-6a74003c6997 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.019527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1c091-1cde-4fbc-a3e8-917d126e3357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.020168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1c091-1cde-4fbc-a3e8-917d126e3357 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.020449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1c091-1cde-4fbc-a3e8-917d126e3357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.020722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd1c091-1cde-4fbc-a3e8-917d126e3357 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.021378 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1c091-1cde-4fbc-a3e8-917d126e3357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.021632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341c3f6-8fed-4ead-8de9-712f4af5b733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.022333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341c3f6-8fed-4ead-8de9-712f4af5b733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.022618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341c3f6-8fed-4ead-8de9-712f4af5b733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.022911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341c3f6-8fed-4ead-8de9-712f4af5b733 admin admin] PATH_INFO: `/v3/users/064ac808853f4133a3e4f3ed26c785ca` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.023308 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd1c091-1cde-4fbc-a3e8-917d126e3357 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.024095 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341c3f6-8fed-4ead-8de9-712f4af5b733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=064ac808853f4133a3e4f3ed26c785ca)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.026041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341c3f6-8fed-4ead-8de9-712f4af5b733 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.052653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1116/2243] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:13:03 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:13:03.057169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1c091-1cde-4fbc-a3e8-917d126e3357 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.088216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8341c3f6-8fed-4ead-8de9-712f4af5b733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '064ac808853f4133a3e4f3ed26c785ca'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:03.089170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8341c3f6-8fed-4ead-8de9-712f4af5b733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '064ac808853f4133a3e4f3ed26c785ca'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:03.089817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8341c3f6-8fed-4ead-8de9-712f4af5b733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '064ac808853f4133a3e4f3ed26c785ca'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:03.103191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8341c3f6-8fed-4ead-8de9-712f4af5b733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '064ac808853f4133a3e4f3ed26c785ca'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:03.110162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8341c3f6-8fed-4ead-8de9-712f4af5b733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '064ac808853f4133a3e4f3ed26c785ca'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:03.124528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1128/2244] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:13:02 2022] DELETE /identity/v3/users/064ac808853f4133a3e4f3ed26c785ca => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:13:03.125760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd1c091-1cde-4fbc-a3e8-917d126e3357 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.130288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701826bd-bc90-4b27-8b36-5159a6e7d1bc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.130573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701826bd-bc90-4b27-8b36-5159a6e7d1bc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.130847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701826bd-bc90-4b27-8b36-5159a6e7d1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.150984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-701826bd-bc90-4b27-8b36-5159a6e7d1bc None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:03.152195 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-701826bd-bc90-4b27-8b36-5159a6e7d1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:03.154763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:03.191880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafa22b-8571-420a-8765-68e9cc890dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.193309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa22b-8571-420a-8765-68e9cc890dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.193598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa22b-8571-420a-8765-68e9cc890dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.193871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cafa22b-8571-420a-8765-68e9cc890dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.194701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafa22b-8571-420a-8765-68e9cc890dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.196663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cafa22b-8571-420a-8765-68e9cc890dd6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.223474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1129/2245] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:03.227651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1117/2246] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:13:03 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:13:03.231497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701826bd-bc90-4b27-8b36-5159a6e7d1bc None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.233202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-701826bd-bc90-4b27-8b36-5159a6e7d1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:03.233572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafa22b-8571-420a-8765-68e9cc890dd6 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.235192 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8cafa22b-8571-420a-8765-68e9cc890dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:03.240482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:03.247055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701826bd-bc90-4b27-8b36-5159a6e7d1bc None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.254389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a66d32-b0bf-434a-8929-6cd2bc00fc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.255015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a66d32-b0bf-434a-8929-6cd2bc00fc57 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.255714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a66d32-b0bf-434a-8929-6cd2bc00fc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.256026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a66d32-b0bf-434a-8929-6cd2bc00fc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.256676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a66d32-b0bf-434a-8929-6cd2bc00fc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.258771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a66d32-b0bf-434a-8929-6cd2bc00fc57 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.271053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1130/2247] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:13:03.275047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a66d32-b0bf-434a-8929-6cd2bc00fc57 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.285304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a66d32-b0bf-434a-8929-6cd2bc00fc57 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.290929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff67721-689b-46cb-9590-b21d961539b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.291514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff67721-689b-46cb-9590-b21d961539b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.291789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff67721-689b-46cb-9590-b21d961539b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.292062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff67721-689b-46cb-9590-b21d961539b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.292690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff67721-689b-46cb-9590-b21d961539b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.294317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff67721-689b-46cb-9590-b21d961539b2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.295038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cafa22b-8571-420a-8765-68e9cc890dd6 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.298779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad87944f-c7bd-4d04-970e-7cf0403cabc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.299382 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87944f-c7bd-4d04-970e-7cf0403cabc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.299664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87944f-c7bd-4d04-970e-7cf0403cabc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.300783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad87944f-c7bd-4d04-970e-7cf0403cabc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.305200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1131/2248] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:13:03.309384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff67721-689b-46cb-9590-b21d961539b2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.311300 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87944f-c7bd-4d04-970e-7cf0403cabc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.312789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad87944f-c7bd-4d04-970e-7cf0403cabc5 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.319154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff67721-689b-46cb-9590-b21d961539b2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.323818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7e256-a806-4cc4-a37c-a2dd0c5d51b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.323973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1118/2249] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:13:03.324451 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7e256-a806-4cc4-a37c-a2dd0c5d51b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.324724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7e256-a806-4cc4-a37c-a2dd0c5d51b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.324992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7e256-a806-4cc4-a37c-a2dd0c5d51b1 admin admin] PATH_INFO: `/v3/projects/d813112767e54019a83d43444901b497/users/8090fe1bdd9742f0840efd3d99ea6eb7/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.340117 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7e256-a806-4cc4-a37c-a2dd0c5d51b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d813112767e54019a83d43444901b497, user_id=8090fe1bdd9742f0840efd3d99ea6eb7, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.342763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7e256-a806-4cc4-a37c-a2dd0c5d51b1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.356255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1132/2250] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:13:03 2022] PUT /identity/v3/projects/d813112767e54019a83d43444901b497/users/8090fe1bdd9742f0840efd3d99ea6eb7/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:13:03.368201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b95671-bf3f-4013-8806-dd6f2ab8b231 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.368742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b95671-bf3f-4013-8806-dd6f2ab8b231 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.369023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b95671-bf3f-4013-8806-dd6f2ab8b231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.399567 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e5b95671-bf3f-4013-8806-dd6f2ab8b231 None None] MFA Rules not processed for user `8090fe1bdd9742f0840efd3d99ea6eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:03.400717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e5b95671-bf3f-4013-8806-dd6f2ab8b231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:03.474955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1119/2251] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:13:03.479482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7e256-a806-4cc4-a37c-a2dd0c5d51b1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.543898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7e256-a806-4cc4-a37c-a2dd0c5d51b1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.548799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68928f2e-102c-4a73-aced-b661ccff481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.549588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68928f2e-102c-4a73-aced-b661ccff481f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.550648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68928f2e-102c-4a73-aced-b661ccff481f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.550923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68928f2e-102c-4a73-aced-b661ccff481f admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.551573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68928f2e-102c-4a73-aced-b661ccff481f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.553527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68928f2e-102c-4a73-aced-b661ccff481f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.582602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1133/2252] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:13:03 2022] POST /identity/v3/projects => generated 383 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:13:03.586580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b95671-bf3f-4013-8806-dd6f2ab8b231 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.598607 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b95671-bf3f-4013-8806-dd6f2ab8b231 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.604750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68928f2e-102c-4a73-aced-b661ccff481f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.606704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-68928f2e-102c-4a73-aced-b661ccff481f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:03.648693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:03.667744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cb3d8-1241-4661-9e1d-4dc5c830f99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.668384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cb3d8-1241-4661-9e1d-4dc5c830f99d admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.668665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cb3d8-1241-4661-9e1d-4dc5c830f99d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.668932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7cb3d8-1241-4661-9e1d-4dc5c830f99d admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.669598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cb3d8-1241-4661-9e1d-4dc5c830f99d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.671564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7cb3d8-1241-4661-9e1d-4dc5c830f99d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.682333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68928f2e-102c-4a73-aced-b661ccff481f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.687712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.688488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.689127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.689445 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] PATH_INFO: `/v3/projects/756d38786c48457194521a27dee60af2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.691442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=756d38786c48457194521a27dee60af2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.693973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.695122 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1120/2253] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:13:03 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:13:03.698915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cb3d8-1241-4661-9e1d-4dc5c830f99d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.709602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7cb3d8-1241-4661-9e1d-4dc5c830f99d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.714295 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ac651-ea48-409e-b127-0d5fdb590452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.714892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ac651-ea48-409e-b127-0d5fdb590452 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.715168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ac651-ea48-409e-b127-0d5fdb590452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.715443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9ac651-ea48-409e-b127-0d5fdb590452 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.716083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ac651-ea48-409e-b127-0d5fdb590452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.717692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9ac651-ea48-409e-b127-0d5fdb590452 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.731478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1121/2254] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:13:03.732474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ac651-ea48-409e-b127-0d5fdb590452 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.743055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9ac651-ea48-409e-b127-0d5fdb590452 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:03.747042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:03.750553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '756d38786c48457194521a27dee60af2', 'request_id': 'req-7445a762-2093-482f-83cb-b33711942596'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:03.762974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '756d38786c48457194521a27dee60af2', 'request_id': 'req-7445a762-2093-482f-83cb-b33711942596'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:03.770523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '756d38786c48457194521a27dee60af2', 'request_id': 'req-7445a762-2093-482f-83cb-b33711942596'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:03.784088 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] The token cache is being invalidate because project 756d38786c48457194521a27dee60af2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:13:03.784476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:03.785330 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:03.786847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1134/2255] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:13:03 2022] DELETE /identity/v3/projects/756d38786c48457194521a27dee60af2 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:03.848450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706fd72-03f7-4d0b-9b87-6dd7af5849d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:03.849293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706fd72-03f7-4d0b-9b87-6dd7af5849d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:03.849578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706fd72-03f7-4d0b-9b87-6dd7af5849d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:03.849943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706fd72-03f7-4d0b-9b87-6dd7af5849d2 admin admin] PATH_INFO: `/v3/projects/3e1d01da56124ac9959756430bbcf6d4/users/5035fc349a134f5b96eeb8265debbeed/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:03.876944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706fd72-03f7-4d0b-9b87-6dd7af5849d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e1d01da56124ac9959756430bbcf6d4, user_id=5035fc349a134f5b96eeb8265debbeed, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:03.880768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706fd72-03f7-4d0b-9b87-6dd7af5849d2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:03.902277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1122/2256] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:13:03 2022] PUT /identity/v3/projects/3e1d01da56124ac9959756430bbcf6d4/users/5035fc349a134f5b96eeb8265debbeed/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:03.907220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:03.908867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:03.931513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445a762-2093-482f-83cb-b33711942596 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:04.004584 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20583da4-b061-4315-8549-7233a8ae0d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:04.005512 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583da4-b061-4315-8549-7233a8ae0d66 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:04.005790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583da4-b061-4315-8549-7233a8ae0d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:04.006058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20583da4-b061-4315-8549-7233a8ae0d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:04.007697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20583da4-b061-4315-8549-7233a8ae0d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:04.009343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20583da4-b061-4315-8549-7233a8ae0d66 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:04.021670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1135/2257] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:13:04.025967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706fd72-03f7-4d0b-9b87-6dd7af5849d2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:04.046378 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706fd72-03f7-4d0b-9b87-6dd7af5849d2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:04.059363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146875b3-4f35-4315-a595-b5c0d0980c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:04.060041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146875b3-4f35-4315-a595-b5c0d0980c88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:04.060321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146875b3-4f35-4315-a595-b5c0d0980c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:04.060590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146875b3-4f35-4315-a595-b5c0d0980c88 admin admin] PATH_INFO: `/v3/projects/3e1d01da56124ac9959756430bbcf6d4/users/5035fc349a134f5b96eeb8265debbeed/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:04.065152 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146875b3-4f35-4315-a595-b5c0d0980c88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e1d01da56124ac9959756430bbcf6d4, user_id=5035fc349a134f5b96eeb8265debbeed, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:04.065556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146875b3-4f35-4315-a595-b5c0d0980c88 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:04.079933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1123/2258] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:13:04 2022] PUT /identity/v3/projects/3e1d01da56124ac9959756430bbcf6d4/users/5035fc349a134f5b96eeb8265debbeed/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:04.090938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627612-1d70-4949-9772-0a95faf52fb9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:04.091226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627612-1d70-4949-9772-0a95faf52fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:04.091496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b627612-1d70-4949-9772-0a95faf52fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:04.120244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7b627612-1d70-4949-9772-0a95faf52fb9 None None] MFA Rules not processed for user `5035fc349a134f5b96eeb8265debbeed`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:04.121490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7b627612-1d70-4949-9772-0a95faf52fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:04.194915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1136/2259] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:04.205346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146875b3-4f35-4315-a595-b5c0d0980c88 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:04.207747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-146875b3-4f35-4315-a595-b5c0d0980c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:04.265611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146875b3-4f35-4315-a595-b5c0d0980c88 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:04.270031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8290c4e-8bf4-49e3-a5ed-5cfc541aba94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:04.270653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8290c4e-8bf4-49e3-a5ed-5cfc541aba94 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:04.270934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8290c4e-8bf4-49e3-a5ed-5cfc541aba94 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:04.271208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8290c4e-8bf4-49e3-a5ed-5cfc541aba94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:04.273022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f8290c4e-8bf4-49e3-a5ed-5cfc541aba94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:04.329761 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:04.334983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8290c4e-8bf4-49e3-a5ed-5cfc541aba94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:04.336854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8290c4e-8bf4-49e3-a5ed-5cfc541aba94 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:04.345956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1124/2260] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:13:11.758915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b627612-1d70-4949-9772-0a95faf52fb9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:11.768314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b627612-1d70-4949-9772-0a95faf52fb9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:11.771847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5c158-c08c-4d34-9544-21fb2b44d25a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:11.772436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5c158-c08c-4d34-9544-21fb2b44d25a service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:11.772708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5c158-c08c-4d34-9544-21fb2b44d25a service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:11.772974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5c158-c08c-4d34-9544-21fb2b44d25a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:11.781613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5c158-c08c-4d34-9544-21fb2b44d25a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:11.782993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5c158-c08c-4d34-9544-21fb2b44d25a service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:11.792598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1137/2261] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:13:14.123739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2e27f-32be-4b8c-bd9f-484a525700c1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:14.124202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2e27f-32be-4b8c-bd9f-484a525700c1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:14.124287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a2e27f-32be-4b8c-bd9f-484a525700c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:14.144443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-65a2e27f-32be-4b8c-bd9f-484a525700c1 None None] MFA Rules not processed for user `ca01fc46c837486b808390ff0c48fa1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:14.145770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-65a2e27f-32be-4b8c-bd9f-484a525700c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:14.200504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1125/2262] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:21.788040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=77,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:23.072734 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5c158-c08c-4d34-9544-21fb2b44d25a service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:23.074255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-46b5c158-c08c-4d34-9544-21fb2b44d25a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:23.165567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5c158-c08c-4d34-9544-21fb2b44d25a service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:23.171908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd6633-4a01-488a-8de5-748b975af943 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:23.172589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd6633-4a01-488a-8de5-748b975af943 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:23.172934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd6633-4a01-488a-8de5-748b975af943 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:23.173301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd6633-4a01-488a-8de5-748b975af943 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:23.175712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b8bd6633-4a01-488a-8de5-748b975af943 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:23.190257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd6633-4a01-488a-8de5-748b975af943 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:23.191998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd6633-4a01-488a-8de5-748b975af943 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:23.201997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1138/2263] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:13:24.194138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=28 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:32.667749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2e27f-32be-4b8c-bd9f-484a525700c1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:32.730973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a2e27f-32be-4b8c-bd9f-484a525700c1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:32.735489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:32.736111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:32.736404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:32.736687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] PATH_INFO: `/v3/users/8090fe1bdd9742f0840efd3d99ea6eb7` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:32.737858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8090fe1bdd9742f0840efd3d99ea6eb7)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:32.739802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:32.789203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8090fe1bdd9742f0840efd3d99ea6eb7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:32.789856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8090fe1bdd9742f0840efd3d99ea6eb7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:32.797754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8090fe1bdd9742f0840efd3d99ea6eb7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:32.798546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8090fe1bdd9742f0840efd3d99ea6eb7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:32.811480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8090fe1bdd9742f0840efd3d99ea6eb7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:32.824384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1126/2264] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:13:32 2022] DELETE /identity/v3/users/8090fe1bdd9742f0840efd3d99ea6eb7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:32.829925 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b764cf-5f9f-4f9b-b5d8-9750dc92e5fa None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:32.830601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b764cf-5f9f-4f9b-b5d8-9750dc92e5fa None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:32.830601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b764cf-5f9f-4f9b-b5d8-9750dc92e5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:32.851179 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-58b764cf-5f9f-4f9b-b5d8-9750dc92e5fa None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:32.852395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-58b764cf-5f9f-4f9b-b5d8-9750dc92e5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:32.913474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1139/2265] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:32.923249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:32.924990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:32.983399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62596018-1584-409a-bbc4-10e41e930562 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:32.987636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cc681-6181-49e3-b1f7-2b6a8906fe88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:32.988352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cc681-6181-49e3-b1f7-2b6a8906fe88 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:32.988814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cc681-6181-49e3-b1f7-2b6a8906fe88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:32.989092 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350cc681-6181-49e3-b1f7-2b6a8906fe88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:32.998963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cc681-6181-49e3-b1f7-2b6a8906fe88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:33.000395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350cc681-6181-49e3-b1f7-2b6a8906fe88 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:33.010416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1127/2266] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:13:33.300646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b764cf-5f9f-4f9b-b5d8-9750dc92e5fa None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:33.302169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-58b764cf-5f9f-4f9b-b5d8-9750dc92e5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:33.314560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b764cf-5f9f-4f9b-b5d8-9750dc92e5fa None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:33.319279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:33.320170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:33.320170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:33.320436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] PATH_INFO: `/v3/projects/d813112767e54019a83d43444901b497` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:33.321586 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d813112767e54019a83d43444901b497)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:33.323557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:33.379995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd813112767e54019a83d43444901b497', 'request_id': 'req-d7ed80b6-a35a-4eb8-9978-bfc549503970'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:33.386125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd813112767e54019a83d43444901b497', 'request_id': 'req-d7ed80b6-a35a-4eb8-9978-bfc549503970'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:33.392388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd813112767e54019a83d43444901b497', 'request_id': 'req-d7ed80b6-a35a-4eb8-9978-bfc549503970'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:33.404636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] The token cache is being invalidate because project d813112767e54019a83d43444901b497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:13:33.404923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:33.405666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:33.407846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1140/2267] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:13:33 2022] DELETE /identity/v3/projects/d813112767e54019a83d43444901b497 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:33.412336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cc681-6181-49e3-b1f7-2b6a8906fe88 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:33.424630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350cc681-6181-49e3-b1f7-2b6a8906fe88 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:33.442982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:33.489985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25abb8d-67c0-40f5-9e56-0d889a6884a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:33.490626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25abb8d-67c0-40f5-9e56-0d889a6884a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:33.490928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25abb8d-67c0-40f5-9e56-0d889a6884a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:33.491195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25abb8d-67c0-40f5-9e56-0d889a6884a4 admin admin] PATH_INFO: `/v3/users/5035fc349a134f5b96eeb8265debbeed` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:33.492314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25abb8d-67c0-40f5-9e56-0d889a6884a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5035fc349a134f5b96eeb8265debbeed)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:33.494259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25abb8d-67c0-40f5-9e56-0d889a6884a4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:33.547946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e25abb8d-67c0-40f5-9e56-0d889a6884a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5035fc349a134f5b96eeb8265debbeed'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:33.548804 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e25abb8d-67c0-40f5-9e56-0d889a6884a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5035fc349a134f5b96eeb8265debbeed'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:33.557144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e25abb8d-67c0-40f5-9e56-0d889a6884a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5035fc349a134f5b96eeb8265debbeed'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:33.557784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e25abb8d-67c0-40f5-9e56-0d889a6884a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5035fc349a134f5b96eeb8265debbeed'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:33.570272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-e25abb8d-67c0-40f5-9e56-0d889a6884a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5035fc349a134f5b96eeb8265debbeed'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:33.589695 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1128/2268] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:13:33 2022] DELETE /identity/v3/users/5035fc349a134f5b96eeb8265debbeed => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:33.866846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:33.868422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:33.883303 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ed80b6-a35a-4eb8-9978-bfc549503970 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:33.893589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:33.947424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:33.947978 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:33.948255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:33.948529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] PATH_INFO: `/v3/projects/3e1d01da56124ac9959756430bbcf6d4` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:33.949701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e1d01da56124ac9959756430bbcf6d4)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:33.951562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:34.002070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e1d01da56124ac9959756430bbcf6d4', 'request_id': 'req-446cdb49-60e5-48af-a0aa-596c76b05d23'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:34.008221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e1d01da56124ac9959756430bbcf6d4', 'request_id': 'req-446cdb49-60e5-48af-a0aa-596c76b05d23'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:34.013744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e1d01da56124ac9959756430bbcf6d4', 'request_id': 'req-446cdb49-60e5-48af-a0aa-596c76b05d23'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:34.027454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] The token cache is being invalidate because project 3e1d01da56124ac9959756430bbcf6d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:13:34.027743 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:34.028389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446cdb49-60e5-48af-a0aa-596c76b05d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:34.029977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1141/2269] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:13:33 2022] DELETE /identity/v3/projects/3e1d01da56124ac9959756430bbcf6d4 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:34.036070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb494a-8375-477f-a3b3-b9f6b623b0b0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.036549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb494a-8375-477f-a3b3-b9f6b623b0b0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.036658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bb494a-8375-477f-a3b3-b9f6b623b0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.056968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d7bb494a-8375-477f-a3b3-b9f6b623b0b0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:34.058253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d7bb494a-8375-477f-a3b3-b9f6b623b0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:34.121769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1129/2270] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:34.132630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860f9c3-8cc9-439c-99b3-0c5e25014a40 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.132918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860f9c3-8cc9-439c-99b3-0c5e25014a40 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.133258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860f9c3-8cc9-439c-99b3-0c5e25014a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.154442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3860f9c3-8cc9-439c-99b3-0c5e25014a40 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:34.155687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3860f9c3-8cc9-439c-99b3-0c5e25014a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:34.163669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1142/2271] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:34.168131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb494a-8375-477f-a3b3-b9f6b623b0b0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:34.179839 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bb494a-8375-477f-a3b3-b9f6b623b0b0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:34.184673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab35edb-13e8-4227-a915-749fa13df820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:34.185324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab35edb-13e8-4227-a915-749fa13df820 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.185637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab35edb-13e8-4227-a915-749fa13df820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.185913 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab35edb-13e8-4227-a915-749fa13df820 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.186563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab35edb-13e8-4227-a915-749fa13df820 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:13:34.186868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab35edb-13e8-4227-a915-749fa13df820 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:34.189170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab35edb-13e8-4227-a915-749fa13df820 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:34.201746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1130/2272] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:13:34.206496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860f9c3-8cc9-439c-99b3-0c5e25014a40 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:34.215977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860f9c3-8cc9-439c-99b3-0c5e25014a40 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:34.220856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfba96e-7f31-4be7-b905-7621e6dea6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:34.221514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfba96e-7f31-4be7-b905-7621e6dea6bf admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.221796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfba96e-7f31-4be7-b905-7621e6dea6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.222066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfba96e-7f31-4be7-b905-7621e6dea6bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.222736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfba96e-7f31-4be7-b905-7621e6dea6bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:34.224612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfba96e-7f31-4be7-b905-7621e6dea6bf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:34.253156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1143/2273] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:13:34 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:13:34.257302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab35edb-13e8-4227-a915-749fa13df820 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:34.268159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab35edb-13e8-4227-a915-749fa13df820 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:34.294321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:34.331093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2e4dc-a094-4288-b7d7-3c873e46b802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:34.332036 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2e4dc-a094-4288-b7d7-3c873e46b802 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.332489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2e4dc-a094-4288-b7d7-3c873e46b802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.332943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b2e4dc-a094-4288-b7d7-3c873e46b802 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.334100 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2e4dc-a094-4288-b7d7-3c873e46b802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:34.337640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b2e4dc-a094-4288-b7d7-3c873e46b802 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:34.363081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1131/2274] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:13:34 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:13:34.369958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfba96e-7f31-4be7-b905-7621e6dea6bf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:34.381905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfba96e-7f31-4be7-b905-7621e6dea6bf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:34.394398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97db8c-e297-470c-8372-4bc476ac5a02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:34.395014 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97db8c-e297-470c-8372-4bc476ac5a02 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.395289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97db8c-e297-470c-8372-4bc476ac5a02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.395557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa97db8c-e297-470c-8372-4bc476ac5a02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.396233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97db8c-e297-470c-8372-4bc476ac5a02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:34.397850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa97db8c-e297-470c-8372-4bc476ac5a02 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:34.409883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1144/2275] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:13:34.413205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2e4dc-a094-4288-b7d7-3c873e46b802 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:34.425547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b2e4dc-a094-4288-b7d7-3c873e46b802 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:34.430572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb93421-596e-425f-b3d0-3dc4b8aea745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:34.431182 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb93421-596e-425f-b3d0-3dc4b8aea745 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.431460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb93421-596e-425f-b3d0-3dc4b8aea745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.431730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb93421-596e-425f-b3d0-3dc4b8aea745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.432361 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb93421-596e-425f-b3d0-3dc4b8aea745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:34.433987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb93421-596e-425f-b3d0-3dc4b8aea745 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:34.444258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1132/2276] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:13:34.449625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97db8c-e297-470c-8372-4bc476ac5a02 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:34.460143 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa97db8c-e297-470c-8372-4bc476ac5a02 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:34.464618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad85ddd-0777-4924-ab6c-825f0ec1527a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:34.465294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad85ddd-0777-4924-ab6c-825f0ec1527a admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.465571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad85ddd-0777-4924-ab6c-825f0ec1527a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.465843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad85ddd-0777-4924-ab6c-825f0ec1527a admin admin] PATH_INFO: `/v3/projects/ce924b7b6c1445f287c548eee2d358ca/users/f6e62b6f7b36464d974df322a6492da2/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.483785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad85ddd-0777-4924-ab6c-825f0ec1527a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce924b7b6c1445f287c548eee2d358ca, user_id=f6e62b6f7b36464d974df322a6492da2, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:34.487601 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad85ddd-0777-4924-ab6c-825f0ec1527a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:34.499977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1145/2277] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:13:34 2022] PUT /identity/v3/projects/ce924b7b6c1445f287c548eee2d358ca/users/f6e62b6f7b36464d974df322a6492da2/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:34.569076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472507e3-6b9c-449e-9a36-48862a697a7e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.569392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472507e3-6b9c-449e-9a36-48862a697a7e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.569669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472507e3-6b9c-449e-9a36-48862a697a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.599448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-472507e3-6b9c-449e-9a36-48862a697a7e None None] MFA Rules not processed for user `f6e62b6f7b36464d974df322a6492da2`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:34.600673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-472507e3-6b9c-449e-9a36-48862a697a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:34.668966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1133/2278] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:34.674371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad85ddd-0777-4924-ab6c-825f0ec1527a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:34.732511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:34.756563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad85ddd-0777-4924-ab6c-825f0ec1527a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:34.762958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f54c43-68a5-490c-a7a6-baec01acf344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:34.762958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f54c43-68a5-490c-a7a6-baec01acf344 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.762958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f54c43-68a5-490c-a7a6-baec01acf344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.762958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f54c43-68a5-490c-a7a6-baec01acf344 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.763365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f54c43-68a5-490c-a7a6-baec01acf344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:34.765000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f54c43-68a5-490c-a7a6-baec01acf344 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:34.800669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1146/2279] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:13:34 2022] POST /identity/v3/projects => generated 385 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:13:34.801873 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472507e3-6b9c-449e-9a36-48862a697a7e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:34.814486 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472507e3-6b9c-449e-9a36-48862a697a7e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:34.875337 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0f54a-d5d6-4bdc-85c1-026f016e8f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:34.876541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0f54a-d5d6-4bdc-85c1-026f016e8f9c admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.876851 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0f54a-d5d6-4bdc-85c1-026f016e8f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.877134 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b0f54a-d5d6-4bdc-85c1-026f016e8f9c admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.877775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0f54a-d5d6-4bdc-85c1-026f016e8f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:34.879669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b0f54a-d5d6-4bdc-85c1-026f016e8f9c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:34.900331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1134/2280] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:13:34 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:13:34.904569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f54c43-68a5-490c-a7a6-baec01acf344 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:34.915095 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f54c43-68a5-490c-a7a6-baec01acf344 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:34.919710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0743c-6ab3-428a-ac3f-e292ad623668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:34.920305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0743c-6ab3-428a-ac3f-e292ad623668 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.920590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0743c-6ab3-428a-ac3f-e292ad623668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.920942 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0743c-6ab3-428a-ac3f-e292ad623668 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.921657 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0743c-6ab3-428a-ac3f-e292ad623668 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:34.923845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0743c-6ab3-428a-ac3f-e292ad623668 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:34.934849 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1147/2281] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:13:34.939661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0f54a-d5d6-4bdc-85c1-026f016e8f9c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:34.949742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b0f54a-d5d6-4bdc-85c1-026f016e8f9c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:34.954696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d7b008-0bb0-4c31-8188-5576286d8fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:34.955280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b008-0bb0-4c31-8188-5576286d8fe2 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.955646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b008-0bb0-4c31-8188-5576286d8fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.955927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d7b008-0bb0-4c31-8188-5576286d8fe2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:34.956564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7b008-0bb0-4c31-8188-5576286d8fe2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:34.958349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d7b008-0bb0-4c31-8188-5576286d8fe2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:34.969622 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1135/2282] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:13:34.974078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0743c-6ab3-428a-ac3f-e292ad623668 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:34.984513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0743c-6ab3-428a-ac3f-e292ad623668 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:34.990562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c852c-9cd0-4d3e-9062-75e7acf4a72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:34.991369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c852c-9cd0-4d3e-9062-75e7acf4a72e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:34.991663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c852c-9cd0-4d3e-9062-75e7acf4a72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:34.991942 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c852c-9cd0-4d3e-9062-75e7acf4a72e admin admin] PATH_INFO: `/v3/projects/8e213982237c4f5aa095d94c0112cb11/users/84421874f1be45e2a060185b460f2b8d/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:35.007901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c852c-9cd0-4d3e-9062-75e7acf4a72e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e213982237c4f5aa095d94c0112cb11, user_id=84421874f1be45e2a060185b460f2b8d, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:35.010570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c852c-9cd0-4d3e-9062-75e7acf4a72e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:35.026581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1148/2283] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:13:34 2022] PUT /identity/v3/projects/8e213982237c4f5aa095d94c0112cb11/users/84421874f1be45e2a060185b460f2b8d/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:35.036834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6378094-c379-451d-ad01-935df5a2aba0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:35.037367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6378094-c379-451d-ad01-935df5a2aba0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:35.037479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6378094-c379-451d-ad01-935df5a2aba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:35.067845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e6378094-c379-451d-ad01-935df5a2aba0 None None] MFA Rules not processed for user `84421874f1be45e2a060185b460f2b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:35.069076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e6378094-c379-451d-ad01-935df5a2aba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:35.072997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:35.136717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1136/2284] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:35.141201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c852c-9cd0-4d3e-9062-75e7acf4a72e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:35.204250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c852c-9cd0-4d3e-9062-75e7acf4a72e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:35.208867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da823cc-2bc9-4dfa-98fb-2b7cd0a53720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:35.209495 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da823cc-2bc9-4dfa-98fb-2b7cd0a53720 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:35.209772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da823cc-2bc9-4dfa-98fb-2b7cd0a53720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:35.210043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da823cc-2bc9-4dfa-98fb-2b7cd0a53720 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:35.210668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da823cc-2bc9-4dfa-98fb-2b7cd0a53720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:35.212549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da823cc-2bc9-4dfa-98fb-2b7cd0a53720 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:35.240251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1149/2285] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:13:35 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:13:35.244065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6378094-c379-451d-ad01-935df5a2aba0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:35.255255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6378094-c379-451d-ad01-935df5a2aba0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:35.311786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179301bb-5575-46fd-8922-511ff4ce9dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:35.312385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179301bb-5575-46fd-8922-511ff4ce9dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:35.312665 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179301bb-5575-46fd-8922-511ff4ce9dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:35.312934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179301bb-5575-46fd-8922-511ff4ce9dd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:35.314138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179301bb-5575-46fd-8922-511ff4ce9dd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:35.316035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179301bb-5575-46fd-8922-511ff4ce9dd4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:35.338492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1137/2286] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:13:35 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:13:35.342571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da823cc-2bc9-4dfa-98fb-2b7cd0a53720 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:35.354076 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da823cc-2bc9-4dfa-98fb-2b7cd0a53720 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:35.358792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9a0d9-b057-4bed-80be-1ee895f1b399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:35.359392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9a0d9-b057-4bed-80be-1ee895f1b399 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:35.359671 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9a0d9-b057-4bed-80be-1ee895f1b399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:35.359945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9a0d9-b057-4bed-80be-1ee895f1b399 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:35.360652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9a0d9-b057-4bed-80be-1ee895f1b399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:35.362301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9a0d9-b057-4bed-80be-1ee895f1b399 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:35.373220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1150/2287] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:13:35.378350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179301bb-5575-46fd-8922-511ff4ce9dd4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:35.388851 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179301bb-5575-46fd-8922-511ff4ce9dd4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:35.394423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10b752-ddd9-4c3d-ab5f-c487457f37de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:35.395072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10b752-ddd9-4c3d-ab5f-c487457f37de admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:35.395353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10b752-ddd9-4c3d-ab5f-c487457f37de admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:35.395624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f10b752-ddd9-4c3d-ab5f-c487457f37de admin admin] PATH_INFO: `/v3/projects/45a6022c2cab475286500283015d0847/users/1d5a15ddbc8f40c6a01282a3008e2c95/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:35.417619 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10b752-ddd9-4c3d-ab5f-c487457f37de admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a6022c2cab475286500283015d0847, user_id=1d5a15ddbc8f40c6a01282a3008e2c95, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:35.420259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f10b752-ddd9-4c3d-ab5f-c487457f37de admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:35.433477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1138/2288] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:13:35 2022] PUT /identity/v3/projects/45a6022c2cab475286500283015d0847/users/1d5a15ddbc8f40c6a01282a3008e2c95/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:35.437547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9a0d9-b057-4bed-80be-1ee895f1b399 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:35.448646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9a0d9-b057-4bed-80be-1ee895f1b399 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:35.515515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c6dcbc-1db5-4564-8a2d-80e0a02891e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:35.516123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6dcbc-1db5-4564-8a2d-80e0a02891e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:35.517042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6dcbc-1db5-4564-8a2d-80e0a02891e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:35.517344 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c6dcbc-1db5-4564-8a2d-80e0a02891e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:35.517987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6dcbc-1db5-4564-8a2d-80e0a02891e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:35.519719 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c6dcbc-1db5-4564-8a2d-80e0a02891e7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:35.525577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:35.532178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1151/2289] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:13:35.535332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10b752-ddd9-4c3d-ab5f-c487457f37de admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:35.545935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f10b752-ddd9-4c3d-ab5f-c487457f37de admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:35.551702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac3f1e-e1ad-421b-920f-b5c2f42fe118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:35.552442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac3f1e-e1ad-421b-920f-b5c2f42fe118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:35.552741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac3f1e-e1ad-421b-920f-b5c2f42fe118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:35.553015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ac3f1e-e1ad-421b-920f-b5c2f42fe118 admin admin] PATH_INFO: `/v3/projects/45a6022c2cab475286500283015d0847/users/1d5a15ddbc8f40c6a01282a3008e2c95/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:35.555043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac3f1e-e1ad-421b-920f-b5c2f42fe118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a6022c2cab475286500283015d0847, user_id=1d5a15ddbc8f40c6a01282a3008e2c95, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:35.557734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ac3f1e-e1ad-421b-920f-b5c2f42fe118 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:35.568561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1139/2290] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:13:35 2022] PUT /identity/v3/projects/45a6022c2cab475286500283015d0847/users/1d5a15ddbc8f40c6a01282a3008e2c95/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:35.580449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f718b6-669f-47b5-90e2-0c180baff5ae None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:35.580763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f718b6-669f-47b5-90e2-0c180baff5ae None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:35.581085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f718b6-669f-47b5-90e2-0c180baff5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:35.611412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-e7f718b6-669f-47b5-90e2-0c180baff5ae None None] MFA Rules not processed for user `1d5a15ddbc8f40c6a01282a3008e2c95`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:35.612659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-e7f718b6-669f-47b5-90e2-0c180baff5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:35.689971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1152/2291] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:35 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:35.698133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac3f1e-e1ad-421b-920f-b5c2f42fe118 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:35.699956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-73ac3f1e-e1ad-421b-920f-b5c2f42fe118 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:35.781835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ac3f1e-e1ad-421b-920f-b5c2f42fe118 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:35.788025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1613ee8-9924-45e4-8dca-1f4a4a82de3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:35.788271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613ee8-9924-45e4-8dca-1f4a4a82de3a service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:35.788548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613ee8-9924-45e4-8dca-1f4a4a82de3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:35.788915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1613ee8-9924-45e4-8dca-1f4a4a82de3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:35.806716 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:35.866253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1613ee8-9924-45e4-8dca-1f4a4a82de3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:35.867683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1613ee8-9924-45e4-8dca-1f4a4a82de3a service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:35.879031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1140/2292] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:13:37.466723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f718b6-669f-47b5-90e2-0c180baff5ae None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:37.476666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f718b6-669f-47b5-90e2-0c180baff5ae None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:37.480186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647304c-7801-465d-bde3-1975285adb01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:37.480788 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647304c-7801-465d-bde3-1975285adb01 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:37.481068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647304c-7801-465d-bde3-1975285adb01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:37.481549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647304c-7801-465d-bde3-1975285adb01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:37.540761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647304c-7801-465d-bde3-1975285adb01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:37.542126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f647304c-7801-465d-bde3-1975285adb01 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:37.552736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1153/2293] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:13:38.190756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1613ee8-9924-45e4-8dca-1f4a4a82de3a service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:38.203023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1613ee8-9924-45e4-8dca-1f4a4a82de3a service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:38.207015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550955a5-113c-41d7-936b-6389ea058e7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:38.207632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550955a5-113c-41d7-936b-6389ea058e7b service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:38.207911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550955a5-113c-41d7-936b-6389ea058e7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:38.208180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550955a5-113c-41d7-936b-6389ea058e7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:38.217661 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550955a5-113c-41d7-936b-6389ea058e7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:38.219085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550955a5-113c-41d7-936b-6389ea058e7b service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:38.237481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1141/2294] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:13:43.241366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647304c-7801-465d-bde3-1975285adb01 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:43.299564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f647304c-7801-465d-bde3-1975285adb01 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:43.304247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:43.304883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:43.305207 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:43.305509 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] PATH_INFO: `/v3/users/f6e62b6f7b36464d974df322a6492da2` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:43.307353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e62b6f7b36464d974df322a6492da2)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:43.309326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:43.340093 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:43.358988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e62b6f7b36464d974df322a6492da2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:43.360547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e62b6f7b36464d974df322a6492da2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:43.361191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e62b6f7b36464d974df322a6492da2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:43.373408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e62b6f7b36464d974df322a6492da2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:43.380667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e62b6f7b36464d974df322a6492da2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:43.394226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1154/2295] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:13:43 2022] DELETE /identity/v3/users/f6e62b6f7b36464d974df322a6492da2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:43.399041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ca20c-e7c4-4f44-938a-74c4ddb268dd None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:43.399538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ca20c-e7c4-4f44-938a-74c4ddb268dd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:43.399633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3ca20c-e7c4-4f44-938a-74c4ddb268dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:43.419068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0f3ca20c-e7c4-4f44-938a-74c4ddb268dd None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:43.420186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0f3ca20c-e7c4-4f44-938a-74c4ddb268dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:43.479076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1142/2296] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:43.489410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:43.490979 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:43.551409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e4cd79-f2ab-49f8-91a2-0aa7da194393 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:43.555066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736152d8-5c9e-41c3-bdda-a576cb52d7f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:43.555669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736152d8-5c9e-41c3-bdda-a576cb52d7f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:43.555946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736152d8-5c9e-41c3-bdda-a576cb52d7f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:43.556216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736152d8-5c9e-41c3-bdda-a576cb52d7f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:43.565410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736152d8-5c9e-41c3-bdda-a576cb52d7f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:43.566833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736152d8-5c9e-41c3-bdda-a576cb52d7f9 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:43.576772 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1155/2297] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:13:43.924475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ca20c-e7c4-4f44-938a-74c4ddb268dd None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:43.926059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0f3ca20c-e7c4-4f44-938a-74c4ddb268dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:43.939541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3ca20c-e7c4-4f44-938a-74c4ddb268dd None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:43.944776 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:43.945520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:43.945801 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:43.946070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] PATH_INFO: `/v3/projects/ce924b7b6c1445f287c548eee2d358ca` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:43.947272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce924b7b6c1445f287c548eee2d358ca)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:43.949252 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:44.011590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce924b7b6c1445f287c548eee2d358ca', 'request_id': 'req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.025273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce924b7b6c1445f287c548eee2d358ca', 'request_id': 'req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.034048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce924b7b6c1445f287c548eee2d358ca', 'request_id': 'req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.049234 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] The token cache is being invalidate because project ce924b7b6c1445f287c548eee2d358ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:13:44.049609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.050410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.058052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1143/2298] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:13:43 2022] DELETE /identity/v3/projects/ce924b7b6c1445f287c548eee2d358ca => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:44.059518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736152d8-5c9e-41c3-bdda-a576cb52d7f9 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:44.076706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736152d8-5c9e-41c3-bdda-a576cb52d7f9 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:44.174307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:44.175991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:44.176418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:44.176761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] PATH_INFO: `/v3/users/84421874f1be45e2a060185b460f2b8d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:44.178607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84421874f1be45e2a060185b460f2b8d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:44.181202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:44.242871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84421874f1be45e2a060185b460f2b8d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.243648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84421874f1be45e2a060185b460f2b8d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.244331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84421874f1be45e2a060185b460f2b8d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.246339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:44.269209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84421874f1be45e2a060185b460f2b8d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.278507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84421874f1be45e2a060185b460f2b8d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.302468 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1156/2299] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:13:44 2022] DELETE /identity/v3/users/84421874f1be45e2a060185b460f2b8d => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:44.618260 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:44.619734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:44.650282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851a4e3f-73ce-4887-a715-fe0a2f5aa60b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:44.672381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:44.714926 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:44.715472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:44.715752 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:44.716025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] PATH_INFO: `/v3/projects/8e213982237c4f5aa095d94c0112cb11` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:44.717188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e213982237c4f5aa095d94c0112cb11)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:44.719868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:44.768104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e213982237c4f5aa095d94c0112cb11', 'request_id': 'req-b49399db-e736-4834-9a9f-46a1d42eadea'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.775366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e213982237c4f5aa095d94c0112cb11', 'request_id': 'req-b49399db-e736-4834-9a9f-46a1d42eadea'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.782302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e213982237c4f5aa095d94c0112cb11', 'request_id': 'req-b49399db-e736-4834-9a9f-46a1d42eadea'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.793708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] The token cache is being invalidate because project 8e213982237c4f5aa095d94c0112cb11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:13:44.794138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.795662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.797509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1144/2300] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:13:44 2022] DELETE /identity/v3/projects/8e213982237c4f5aa095d94c0112cb11 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:44.802879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:44.804663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:44.818022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b64c3b-95dd-40f5-a3af-19c6e1aab439 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:44.880367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31628fa-9d6f-4e46-99cf-5f1df876f25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:44.881002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31628fa-9d6f-4e46-99cf-5f1df876f25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:44.881310 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31628fa-9d6f-4e46-99cf-5f1df876f25f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:44.881582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31628fa-9d6f-4e46-99cf-5f1df876f25f admin admin] PATH_INFO: `/v3/users/1d5a15ddbc8f40c6a01282a3008e2c95` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:44.882703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31628fa-9d6f-4e46-99cf-5f1df876f25f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d5a15ddbc8f40c6a01282a3008e2c95)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:44.884626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31628fa-9d6f-4e46-99cf-5f1df876f25f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:44.929761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f31628fa-9d6f-4e46-99cf-5f1df876f25f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d5a15ddbc8f40c6a01282a3008e2c95'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.930425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f31628fa-9d6f-4e46-99cf-5f1df876f25f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d5a15ddbc8f40c6a01282a3008e2c95'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.931030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f31628fa-9d6f-4e46-99cf-5f1df876f25f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d5a15ddbc8f40c6a01282a3008e2c95'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.943651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f31628fa-9d6f-4e46-99cf-5f1df876f25f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d5a15ddbc8f40c6a01282a3008e2c95'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.950532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-f31628fa-9d6f-4e46-99cf-5f1df876f25f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d5a15ddbc8f40c6a01282a3008e2c95'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:44.963549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1157/2301] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:13:44 2022] DELETE /identity/v3/users/1d5a15ddbc8f40c6a01282a3008e2c95 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:13:45.229559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:45.231072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:45.245041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49399db-e736-4834-9a9f-46a1d42eadea admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:45.302934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:45.303583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:45.303861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:45.304125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] PATH_INFO: `/v3/projects/45a6022c2cab475286500283015d0847` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:45.305291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a6022c2cab475286500283015d0847)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:45.307173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:45.356417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a6022c2cab475286500283015d0847', 'request_id': 'req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:45.361620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:45.364994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a6022c2cab475286500283015d0847', 'request_id': 'req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:45.373374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a6022c2cab475286500283015d0847', 'request_id': 'req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:45.385439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] The token cache is being invalidate because project 45a6022c2cab475286500283015d0847 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:13:45.385734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:45.386383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f3d2fd29-e3d2-4be4-be69-81b2a2aac7ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:13:45.388124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1145/2302] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:13:45 2022] DELETE /identity/v3/projects/45a6022c2cab475286500283015d0847 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:45.392879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aef92-cb91-413d-a40c-abfdbcecdd62 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:45.393462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aef92-cb91-413d-a40c-abfdbcecdd62 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:45.393737 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4aef92-cb91-413d-a40c-abfdbcecdd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:45.413764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-fe4aef92-cb91-413d-a40c-abfdbcecdd62 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:45.414968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fe4aef92-cb91-413d-a40c-abfdbcecdd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:45.456408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:45.477706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1158/2303] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:45.485660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c025c-4184-462e-b96e-65081399852f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:45.485948 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c025c-4184-462e-b96e-65081399852f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:45.486225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7c025c-4184-462e-b96e-65081399852f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:45.505532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3a7c025c-4184-462e-b96e-65081399852f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:45.506760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3a7c025c-4184-462e-b96e-65081399852f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:45.514623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1146/2304] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:45.519318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4aef92-cb91-413d-a40c-abfdbcecdd62 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:45.528738 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4aef92-cb91-413d-a40c-abfdbcecdd62 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:45.533898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75bb4-ea06-4750-b478-5dee72873d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:45.534488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75bb4-ea06-4750-b478-5dee72873d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:45.534766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75bb4-ea06-4750-b478-5dee72873d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:45.535040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e75bb4-ea06-4750-b478-5dee72873d0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:45.535710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e75bb4-ea06-4750-b478-5dee72873d0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:13:45.536012 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e75bb4-ea06-4750-b478-5dee72873d0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:45.537623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e75bb4-ea06-4750-b478-5dee72873d0e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:45.549240 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1159/2305] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:13:45.553394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c025c-4184-462e-b96e-65081399852f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:45.562367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7c025c-4184-462e-b96e-65081399852f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:45.567300 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ddf58-8f15-4116-be1e-873f48d67088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:45.567881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ddf58-8f15-4116-be1e-873f48d67088 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:45.568158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ddf58-8f15-4116-be1e-873f48d67088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:45.568425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ddf58-8f15-4116-be1e-873f48d67088 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:45.569046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ddf58-8f15-4116-be1e-873f48d67088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:45.570967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7ddf58-8f15-4116-be1e-873f48d67088 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:45.598300 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1147/2306] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:13:45 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:13:45.602483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75bb4-ea06-4750-b478-5dee72873d0e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:45.612441 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e75bb4-ea06-4750-b478-5dee72873d0e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:45.667846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc32c3-979d-42d3-9720-1ab6788f3caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:45.668443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc32c3-979d-42d3-9720-1ab6788f3caa admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:45.668721 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc32c3-979d-42d3-9720-1ab6788f3caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:45.668992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc32c3-979d-42d3-9720-1ab6788f3caa admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:45.669663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc32c3-979d-42d3-9720-1ab6788f3caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:45.671551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bc32c3-979d-42d3-9720-1ab6788f3caa admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:45.697241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1160/2307] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:13:45 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:13:45.698409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ddf58-8f15-4116-be1e-873f48d67088 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:45.709734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7ddf58-8f15-4116-be1e-873f48d67088 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:45.714300 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc79933-b8bb-439d-ac2f-1334e02604ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:45.714922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc79933-b8bb-439d-ac2f-1334e02604ca admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:45.715196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc79933-b8bb-439d-ac2f-1334e02604ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:45.715469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc79933-b8bb-439d-ac2f-1334e02604ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:45.716105 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc79933-b8bb-439d-ac2f-1334e02604ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:45.717755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc79933-b8bb-439d-ac2f-1334e02604ca admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:45.727525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1148/2308] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:13:45.731467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc32c3-979d-42d3-9720-1ab6788f3caa admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:45.741447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc32c3-979d-42d3-9720-1ab6788f3caa admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:45.746087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39efd49b-a262-46dd-9a56-da08c9a5c5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:45.746680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efd49b-a262-46dd-9a56-da08c9a5c5cd admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:45.746959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efd49b-a262-46dd-9a56-da08c9a5c5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:45.747229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39efd49b-a262-46dd-9a56-da08c9a5c5cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:45.747871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efd49b-a262-46dd-9a56-da08c9a5c5cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:45.749512 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39efd49b-a262-46dd-9a56-da08c9a5c5cd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:45.760404 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1161/2309] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:13:45.764489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc79933-b8bb-439d-ac2f-1334e02604ca admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:45.775108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc79933-b8bb-439d-ac2f-1334e02604ca admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:45.779726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8741b-9365-42b2-9a27-3702d467e9c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:45.780354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8741b-9365-42b2-9a27-3702d467e9c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:45.780632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8741b-9365-42b2-9a27-3702d467e9c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:45.780904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8741b-9365-42b2-9a27-3702d467e9c4 admin admin] PATH_INFO: `/v3/projects/11b30bb85e764136ad70f6649f26f9ba/users/ecdab9ce0a404f09a153eea9bcdee7f5/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:45.795168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8741b-9365-42b2-9a27-3702d467e9c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11b30bb85e764136ad70f6649f26f9ba, user_id=ecdab9ce0a404f09a153eea9bcdee7f5, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:45.797944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f8741b-9365-42b2-9a27-3702d467e9c4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:45.810718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1149/2310] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:13:45 2022] PUT /identity/v3/projects/11b30bb85e764136ad70f6649f26f9ba/users/ecdab9ce0a404f09a153eea9bcdee7f5/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:13:45.822654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2b042-afc5-4e13-be00-9e7d0bbb774d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:45.822943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2b042-afc5-4e13-be00-9e7d0bbb774d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:45.823216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2b042-afc5-4e13-be00-9e7d0bbb774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:45.850669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-66a2b042-afc5-4e13-be00-9e7d0bbb774d None None] MFA Rules not processed for user `ecdab9ce0a404f09a153eea9bcdee7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:45.851882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-66a2b042-afc5-4e13-be00-9e7d0bbb774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:45.914738 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1162/2311] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:13:45.926515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8741b-9365-42b2-9a27-3702d467e9c4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:13:45.928186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c1f8741b-9365-42b2-9a27-3702d467e9c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:45.983974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8741b-9365-42b2-9a27-3702d467e9c4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:13:45.987597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f63a6-c583-48c2-8590-2689fe71bc86 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '195565a7695b4d9b82e0ee76181fee2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:13:45.988190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f63a6-c583-48c2-8590-2689fe71bc86 service glance] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:45.988467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f63a6-c583-48c2-8590-2689fe71bc86 service glance] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:45.988739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98f63a6-c583-48c2-8590-2689fe71bc86 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:45.996788 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f63a6-c583-48c2-8590-2689fe71bc86 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:13:45.998386 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98f63a6-c583-48c2-8590-2689fe71bc86 service glance] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:13:46.007842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1150/2312] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:13:45 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:13:55.911846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=79,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:56.003873 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=59,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:13:59.116373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10ea80-5a95-49dc-a609-3675d7d9ada3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:13:59.116957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10ea80-5a95-49dc-a609-3675d7d9ada3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:13:59.117342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f10ea80-5a95-49dc-a609-3675d7d9ada3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:13:59.139229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3f10ea80-5a95-49dc-a609-3675d7d9ada3 None None] MFA Rules not processed for user `cd913a3c0cff4787a90603f3afb2e883`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:13:59.140577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3f10ea80-5a95-49dc-a609-3675d7d9ada3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:13:59.197991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1163/2313] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:13:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:06.483626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f63a6-c583-48c2-8590-2689fe71bc86 service glance] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:06.485450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c98f63a6-c583-48c2-8590-2689fe71bc86 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:06.564749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98f63a6-c583-48c2-8590-2689fe71bc86 service glance] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:06.569512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:06.570137 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:06.570410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:06.570677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] PATH_INFO: `/v3/users/9c54c3844bc0421fbca966af7195843d` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:06.571783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c54c3844bc0421fbca966af7195843d)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:06.573744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:06.622289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c54c3844bc0421fbca966af7195843d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:06.622939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c54c3844bc0421fbca966af7195843d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:06.630428 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c54c3844bc0421fbca966af7195843d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:06.631072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c54c3844bc0421fbca966af7195843d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:06.643285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c54c3844bc0421fbca966af7195843d'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:06.655664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1151/2314] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:14:06 2022] DELETE /identity/v3/users/9c54c3844bc0421fbca966af7195843d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:14:06.660097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45305d93-2d81-4d28-929c-e17f127ef20d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:06.660618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45305d93-2d81-4d28-929c-e17f127ef20d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:06.660618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45305d93-2d81-4d28-929c-e17f127ef20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:06.679316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-45305d93-2d81-4d28-929c-e17f127ef20d None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:06.680497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-45305d93-2d81-4d28-929c-e17f127ef20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:06.741002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1164/2315] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:06.751801 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:06.753223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:06.813952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1b412-6dde-4183-82ea-3378fc740e27 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:06.817606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68832ba6-c39a-4d9d-98b2-502004615692 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:06.818321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68832ba6-c39a-4d9d-98b2-502004615692 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:06.818605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68832ba6-c39a-4d9d-98b2-502004615692 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:06.818874 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68832ba6-c39a-4d9d-98b2-502004615692 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:06.829002 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68832ba6-c39a-4d9d-98b2-502004615692 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:06.830584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68832ba6-c39a-4d9d-98b2-502004615692 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:06.840179 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1152/2316] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:14:07.103294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45305d93-2d81-4d28-929c-e17f127ef20d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:07.104744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-45305d93-2d81-4d28-929c-e17f127ef20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:07.120470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45305d93-2d81-4d28-929c-e17f127ef20d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:07.126565 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:07.127238 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:07.127506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:07.127812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] PATH_INFO: `/v3/projects/13b34455f657497ea9cef0305699e724` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:07.128933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13b34455f657497ea9cef0305699e724)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:07.130871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:07.177539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13b34455f657497ea9cef0305699e724', 'request_id': 'req-634bb812-aa77-4527-bbdf-a22cbc114898'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:07.183550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13b34455f657497ea9cef0305699e724', 'request_id': 'req-634bb812-aa77-4527-bbdf-a22cbc114898'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:07.189707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13b34455f657497ea9cef0305699e724', 'request_id': 'req-634bb812-aa77-4527-bbdf-a22cbc114898'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:07.201818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] The token cache is being invalidate because project 13b34455f657497ea9cef0305699e724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:14:07.202106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:07.202895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-634bb812-aa77-4527-bbdf-a22cbc114898 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:07.204390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1165/2317] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:14:07 2022] DELETE /identity/v3/projects/13b34455f657497ea9cef0305699e724 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:14:07.209349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc26756-f852-4c1c-bb65-b7ed20a6dd1b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:07.209641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc26756-f852-4c1c-bb65-b7ed20a6dd1b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:07.209914 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc26756-f852-4c1c-bb65-b7ed20a6dd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:07.230629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6cc26756-f852-4c1c-bb65-b7ed20a6dd1b None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:07.231759 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6cc26756-f852-4c1c-bb65-b7ed20a6dd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:07.250113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:14:07.293986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1153/2318] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:07.303921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6ad90-2c3b-479a-8514-ef4af1f201a3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:07.304300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6ad90-2c3b-479a-8514-ef4af1f201a3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:07.304643 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6ad90-2c3b-479a-8514-ef4af1f201a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:07.325067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-63c6ad90-2c3b-479a-8514-ef4af1f201a3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:07.326331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-63c6ad90-2c3b-479a-8514-ef4af1f201a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:07.334488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1166/2319] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:14:07.338709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc26756-f852-4c1c-bb65-b7ed20a6dd1b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:07.348364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc26756-f852-4c1c-bb65-b7ed20a6dd1b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:07.352961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61cd15-7aa2-4aa2-a3db-76b2f3cec33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:07.353597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61cd15-7aa2-4aa2-a3db-76b2f3cec33d admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:07.353876 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61cd15-7aa2-4aa2-a3db-76b2f3cec33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:07.354145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a61cd15-7aa2-4aa2-a3db-76b2f3cec33d admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:07.354785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61cd15-7aa2-4aa2-a3db-76b2f3cec33d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:14:07.355097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61cd15-7aa2-4aa2-a3db-76b2f3cec33d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:07.356690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a61cd15-7aa2-4aa2-a3db-76b2f3cec33d admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:07.368698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1154/2320] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:14:07.373015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6ad90-2c3b-479a-8514-ef4af1f201a3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:07.382957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6ad90-2c3b-479a-8514-ef4af1f201a3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:07.387845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979554ca-3674-4fb8-b684-edee93d0c611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:07.388516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979554ca-3674-4fb8-b684-edee93d0c611 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:07.388857 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979554ca-3674-4fb8-b684-edee93d0c611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:07.389166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979554ca-3674-4fb8-b684-edee93d0c611 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:07.389839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979554ca-3674-4fb8-b684-edee93d0c611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:07.391809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979554ca-3674-4fb8-b684-edee93d0c611 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:07.414107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:14:07.423180 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1167/2321] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:14:07 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:14:07.426650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61cd15-7aa2-4aa2-a3db-76b2f3cec33d admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:07.437119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a61cd15-7aa2-4aa2-a3db-76b2f3cec33d admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:07.495229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40c3f4-eed3-464b-b86a-e53e412dcb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:07.495832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40c3f4-eed3-464b-b86a-e53e412dcb91 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:07.496114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40c3f4-eed3-464b-b86a-e53e412dcb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:07.496389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40c3f4-eed3-464b-b86a-e53e412dcb91 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:07.497031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40c3f4-eed3-464b-b86a-e53e412dcb91 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:07.498966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40c3f4-eed3-464b-b86a-e53e412dcb91 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:07.521005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1155/2322] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:14:07 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:14:07.525400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979554ca-3674-4fb8-b684-edee93d0c611 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:07.537024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979554ca-3674-4fb8-b684-edee93d0c611 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:07.542003 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2f4ae-714d-403f-bae0-cc60280aebab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:07.542670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2f4ae-714d-403f-bae0-cc60280aebab admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:07.542952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2f4ae-714d-403f-bae0-cc60280aebab admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:07.543221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2f4ae-714d-403f-bae0-cc60280aebab admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:07.543886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2f4ae-714d-403f-bae0-cc60280aebab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:07.545780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2f4ae-714d-403f-bae0-cc60280aebab admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:07.559454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1168/2323] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:14:07.560856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40c3f4-eed3-464b-b86a-e53e412dcb91 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:07.571307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40c3f4-eed3-464b-b86a-e53e412dcb91 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:07.575955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073581c-3067-4c88-9dd8-c84056f785bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:07.576536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073581c-3067-4c88-9dd8-c84056f785bd admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:07.576813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073581c-3067-4c88-9dd8-c84056f785bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:07.577083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073581c-3067-4c88-9dd8-c84056f785bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:07.578374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073581c-3067-4c88-9dd8-c84056f785bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:07.579962 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073581c-3067-4c88-9dd8-c84056f785bd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:07.590238 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1156/2324] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:14:07.594480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2f4ae-714d-403f-bae0-cc60280aebab admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:07.604336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2f4ae-714d-403f-bae0-cc60280aebab admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:07.609447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5d5c6-9523-4896-b574-66e49da57372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:07.610103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5d5c6-9523-4896-b574-66e49da57372 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:07.610382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5d5c6-9523-4896-b574-66e49da57372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:07.610656 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa5d5c6-9523-4896-b574-66e49da57372 admin admin] PATH_INFO: `/v3/projects/322c7c4ad81c41dfb148ae965fc964dc/users/ce2f4de513274a00bd9a8783a7ff358a/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:07.625973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5d5c6-9523-4896-b574-66e49da57372 admin admin] RBAC: Authorizing `identity:create_grant(project_id=322c7c4ad81c41dfb148ae965fc964dc, user_id=ce2f4de513274a00bd9a8783a7ff358a, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:07.628624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa5d5c6-9523-4896-b574-66e49da57372 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:07.642487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1169/2325] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:14:07 2022] PUT /identity/v3/projects/322c7c4ad81c41dfb148ae965fc964dc/users/ce2f4de513274a00bd9a8783a7ff358a/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:14:07.652658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab343af5-d1f2-4c93-a2bf-4a78d115daf8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:07.652967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab343af5-d1f2-4c93-a2bf-4a78d115daf8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:07.653289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab343af5-d1f2-4c93-a2bf-4a78d115daf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:07.681049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ab343af5-d1f2-4c93-a2bf-4a78d115daf8 None None] MFA Rules not processed for user `ce2f4de513274a00bd9a8783a7ff358a`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:07.682400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ab343af5-d1f2-4c93-a2bf-4a78d115daf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:07.747066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1157/2326] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:14:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:07.758419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5d5c6-9523-4896-b574-66e49da57372 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:07.760201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1fa5d5c6-9523-4896-b574-66e49da57372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:07.818576 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa5d5c6-9523-4896-b574-66e49da57372 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:07.822260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201bc13-d257-44a5-a866-f36c978a73ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:07.822863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201bc13-d257-44a5-a866-f36c978a73ce service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:07.823140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201bc13-d257-44a5-a866-f36c978a73ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:07.823412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a201bc13-d257-44a5-a866-f36c978a73ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:07.832066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201bc13-d257-44a5-a866-f36c978a73ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:07.833520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a201bc13-d257-44a5-a866-f36c978a73ce service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:07.842413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1170/2327] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:14:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:14:10.279239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44989e-c2ed-43e9-ae25-24427cbeffca None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:10.279809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44989e-c2ed-43e9-ae25-24427cbeffca None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:10.279809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44989e-c2ed-43e9-ae25-24427cbeffca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:10.299126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6e44989e-c2ed-43e9-ae25-24427cbeffca None None] MFA Rules not processed for user `ce2f4de513274a00bd9a8783a7ff358a`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:10.300274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6e44989e-c2ed-43e9-ae25-24427cbeffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:10.307259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1158/2328] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:14:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:17.838409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:14:20.293768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:14:44.171871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201bc13-d257-44a5-a866-f36c978a73ce service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:44.173624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a201bc13-d257-44a5-a866-f36c978a73ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:44.239480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a201bc13-d257-44a5-a866-f36c978a73ce service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:44.244283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:44.245121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:44.245418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:44.245691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] PATH_INFO: `/v3/users/ecdab9ce0a404f09a153eea9bcdee7f5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:44.247910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecdab9ce0a404f09a153eea9bcdee7f5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:44.249859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:44.298130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecdab9ce0a404f09a153eea9bcdee7f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:44.298785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecdab9ce0a404f09a153eea9bcdee7f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:44.299390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecdab9ce0a404f09a153eea9bcdee7f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:44.312602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecdab9ce0a404f09a153eea9bcdee7f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:44.319761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecdab9ce0a404f09a153eea9bcdee7f5'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:44.333300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1171/2329] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:14:44 2022] DELETE /identity/v3/users/ecdab9ce0a404f09a153eea9bcdee7f5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:14:44.337864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdf183-3e9d-46af-87c2-46825cc7cac9 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:44.338395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdf183-3e9d-46af-87c2-46825cc7cac9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:44.338556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acdf183-3e9d-46af-87c2-46825cc7cac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:44.359822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8acdf183-3e9d-46af-87c2-46825cc7cac9 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:44.361158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8acdf183-3e9d-46af-87c2-46825cc7cac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:44.442391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1159/2330] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:44.449061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:44.451322 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:44.532398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888c7f19-0943-4f86-8d47-a5cd647d759f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:44.536260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a40df-051d-45dd-b7cb-22587017b706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:44.536986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a40df-051d-45dd-b7cb-22587017b706 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:44.537523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a40df-051d-45dd-b7cb-22587017b706 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:44.537930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a40df-051d-45dd-b7cb-22587017b706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:44.548558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a40df-051d-45dd-b7cb-22587017b706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:44.550200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a40df-051d-45dd-b7cb-22587017b706 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:44.578202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1172/2331] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:14:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:14:44.873783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdf183-3e9d-46af-87c2-46825cc7cac9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:44.875361 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8acdf183-3e9d-46af-87c2-46825cc7cac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:44.888697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acdf183-3e9d-46af-87c2-46825cc7cac9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:44.893404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:44.894031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:44.894309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:44.894579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] PATH_INFO: `/v3/projects/11b30bb85e764136ad70f6649f26f9ba` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:44.895696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11b30bb85e764136ad70f6649f26f9ba)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:44.897621 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:44.944961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11b30bb85e764136ad70f6649f26f9ba', 'request_id': 'req-391c8a94-8f07-4a80-8d42-edc4c33285e3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:44.951939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11b30bb85e764136ad70f6649f26f9ba', 'request_id': 'req-391c8a94-8f07-4a80-8d42-edc4c33285e3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:44.958276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11b30bb85e764136ad70f6649f26f9ba', 'request_id': 'req-391c8a94-8f07-4a80-8d42-edc4c33285e3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:44.968772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] The token cache is being invalidate because project 11b30bb85e764136ad70f6649f26f9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:14:44.969141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:44.969782 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:44.971303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1160/2332] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:14:44 2022] DELETE /identity/v3/projects/11b30bb85e764136ad70f6649f26f9ba => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:14:46.496735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff179f-dae2-4d35-b82c-7ba5329df096 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:46.497201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff179f-dae2-4d35-b82c-7ba5329df096 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:46.497327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dff179f-dae2-4d35-b82c-7ba5329df096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:46.516633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5dff179f-dae2-4d35-b82c-7ba5329df096 None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:46.517855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5dff179f-dae2-4d35-b82c-7ba5329df096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:46.536029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:14:46.582689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1173/2333] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:14:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:46.629944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:46.631551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:46.705401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391c8a94-8f07-4a80-8d42-edc4c33285e3 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:46.710484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e5c3f-df2f-47c6-9992-1ca738cac2df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:46.711081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e5c3f-df2f-47c6-9992-1ca738cac2df service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:46.711358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e5c3f-df2f-47c6-9992-1ca738cac2df service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:46.711630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e5c3f-df2f-47c6-9992-1ca738cac2df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:46.719609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e5c3f-df2f-47c6-9992-1ca738cac2df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:46.721214 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e5c3f-df2f-47c6-9992-1ca738cac2df service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:46.729593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1161/2334] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:14:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:14:47.306201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfeed7-333a-48b3-871f-21c4dd8a9ad8 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:47.306486 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfeed7-333a-48b3-871f-21c4dd8a9ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:47.306758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfeed7-333a-48b3-871f-21c4dd8a9ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:47.329198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-95bfeed7-333a-48b3-871f-21c4dd8a9ad8 None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:47.330395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-95bfeed7-333a-48b3-871f-21c4dd8a9ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:47.337526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1174/2335] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 22 22:14:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:47.350604 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e5c3f-df2f-47c6-9992-1ca738cac2df service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:47.360889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e5c3f-df2f-47c6-9992-1ca738cac2df service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:47.365848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53545a5-bda1-4969-8ce7-89ada4ba5915 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:47.366424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53545a5-bda1-4969-8ce7-89ada4ba5915 service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:47.366699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53545a5-bda1-4969-8ce7-89ada4ba5915 service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:47.366965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53545a5-bda1-4969-8ce7-89ada4ba5915 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:47.374693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53545a5-bda1-4969-8ce7-89ada4ba5915 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:47.376243 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53545a5-bda1-4969-8ce7-89ada4ba5915 service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:47.380611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:14:47.384422 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1162/2336] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:14:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:14:47.639207 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b63fea-284b-4d2b-8c80-2276ebb765ef None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:47.639674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b63fea-284b-4d2b-8c80-2276ebb765ef None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:47.639773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b63fea-284b-4d2b-8c80-2276ebb765ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:47.658627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-33b63fea-284b-4d2b-8c80-2276ebb765ef None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:47.659815 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-33b63fea-284b-4d2b-8c80-2276ebb765ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:47.666826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1175/2337] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:14:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:14:47.678931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53545a5-bda1-4969-8ce7-89ada4ba5915 service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:47.690009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53545a5-bda1-4969-8ce7-89ada4ba5915 service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:47.694965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db3c7a-79f7-41d6-a127-e85d83eb2566 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:47.695570 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db3c7a-79f7-41d6-a127-e85d83eb2566 service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:47.695830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db3c7a-79f7-41d6-a127-e85d83eb2566 service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:47.696099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db3c7a-79f7-41d6-a127-e85d83eb2566 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:47.704640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db3c7a-79f7-41d6-a127-e85d83eb2566 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:47.706238 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db3c7a-79f7-41d6-a127-e85d83eb2566 service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:47.714067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1163/2338] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:14:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:14:48.033348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0de95a-c1a5-4f78-bc76-54712c0eba92 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:48.033629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0de95a-c1a5-4f78-bc76-54712c0eba92 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:48.033897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0de95a-c1a5-4f78-bc76-54712c0eba92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:48.052924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-9b0de95a-c1a5-4f78-bc76-54712c0eba92 None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:48.054152 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9b0de95a-c1a5-4f78-bc76-54712c0eba92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:48.061421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1176/2339] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 22 22:14:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:48.074013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db3c7a-79f7-41d6-a127-e85d83eb2566 service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:48.084969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db3c7a-79f7-41d6-a127-e85d83eb2566 service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:48.090480 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f72281-6ec0-4aa5-92c8-ea76a390d65c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:48.091247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f72281-6ec0-4aa5-92c8-ea76a390d65c service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:48.091522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f72281-6ec0-4aa5-92c8-ea76a390d65c service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:48.091793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f72281-6ec0-4aa5-92c8-ea76a390d65c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:48.099609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f72281-6ec0-4aa5-92c8-ea76a390d65c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:48.101189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f72281-6ec0-4aa5-92c8-ea76a390d65c service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:48.108986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1164/2340] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:14:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:14:50.200254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0de95a-c1a5-4f78-bc76-54712c0eba92 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:50.201715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9b0de95a-c1a5-4f78-bc76-54712c0eba92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:50.269487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0de95a-c1a5-4f78-bc76-54712c0eba92 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:50.274069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:50.274712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:50.274992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:50.275266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] PATH_INFO: `/v3/users/ce2f4de513274a00bd9a8783a7ff358a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:50.276409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce2f4de513274a00bd9a8783a7ff358a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:50.278457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:50.326482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce2f4de513274a00bd9a8783a7ff358a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:50.327324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce2f4de513274a00bd9a8783a7ff358a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:50.328638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce2f4de513274a00bd9a8783a7ff358a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:50.340800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce2f4de513274a00bd9a8783a7ff358a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:50.347673 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce2f4de513274a00bd9a8783a7ff358a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:50.361794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1177/2341] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:14:50 2022] DELETE /identity/v3/users/ce2f4de513274a00bd9a8783a7ff358a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:14:50.366868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca90a29-94f0-4980-845a-ea6ed46aa98e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:50.367346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca90a29-94f0-4980-845a-ea6ed46aa98e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:50.367570 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca90a29-94f0-4980-845a-ea6ed46aa98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:50.387813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-bca90a29-94f0-4980-845a-ea6ed46aa98e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:50.389013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bca90a29-94f0-4980-845a-ea6ed46aa98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:50.450873 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1165/2342] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:50.460596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:50.462024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:50.493901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:14:50.522620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43aa562-638d-453d-8d91-03bed46de100 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:50.526402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6144df-f29a-4979-a2cd-f231306df6d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:50.527008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6144df-f29a-4979-a2cd-f231306df6d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:50.527919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6144df-f29a-4979-a2cd-f231306df6d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:50.528200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6144df-f29a-4979-a2cd-f231306df6d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:50.537215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6144df-f29a-4979-a2cd-f231306df6d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:50.538667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6144df-f29a-4979-a2cd-f231306df6d9 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:50.548265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1178/2343] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:14:50.715735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca90a29-94f0-4980-845a-ea6ed46aa98e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:50.717550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bca90a29-94f0-4980-845a-ea6ed46aa98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:50.730117 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca90a29-94f0-4980-845a-ea6ed46aa98e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:50.734834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:50.735735 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:50.736074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:50.736563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] PATH_INFO: `/v3/projects/322c7c4ad81c41dfb148ae965fc964dc` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:50.737975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=322c7c4ad81c41dfb148ae965fc964dc)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:50.740063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:50.794636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322c7c4ad81c41dfb148ae965fc964dc', 'request_id': 'req-cc825027-fbe3-4896-8026-237fc95ad688'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:50.803814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322c7c4ad81c41dfb148ae965fc964dc', 'request_id': 'req-cc825027-fbe3-4896-8026-237fc95ad688'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:50.815547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322c7c4ad81c41dfb148ae965fc964dc', 'request_id': 'req-cc825027-fbe3-4896-8026-237fc95ad688'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:50.828842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] The token cache is being invalidate because project 322c7c4ad81c41dfb148ae965fc964dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:14:50.829296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:50.830075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-cc825027-fbe3-4896-8026-237fc95ad688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:14:50.831889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1166/2344] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:14:50 2022] DELETE /identity/v3/projects/322c7c4ad81c41dfb148ae965fc964dc => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:14:50.837145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6193f6c-26bc-48fe-8064-3b725a8f7109 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:50.837411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6193f6c-26bc-48fe-8064-3b725a8f7109 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:50.837689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6193f6c-26bc-48fe-8064-3b725a8f7109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:50.863797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b6193f6c-26bc-48fe-8064-3b725a8f7109 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:50.865033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b6193f6c-26bc-48fe-8064-3b725a8f7109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:50.928856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1179/2345] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:14:50.937497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ca921-295f-4fd9-95d7-c0bc29f00d79 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:50.937854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ca921-295f-4fd9-95d7-c0bc29f00d79 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:50.938178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ca921-295f-4fd9-95d7-c0bc29f00d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:50.957795 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-463ca921-295f-4fd9-95d7-c0bc29f00d79 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:50.959003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-463ca921-295f-4fd9-95d7-c0bc29f00d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:50.967125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1167/2346] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:14:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:50.971253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6193f6c-26bc-48fe-8064-3b725a8f7109 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:50.982144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6193f6c-26bc-48fe-8064-3b725a8f7109 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:50.987717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5a5a0-649f-409d-8a34-48f25fc16ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:50.988313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5a5a0-649f-409d-8a34-48f25fc16ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:50.988597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5a5a0-649f-409d-8a34-48f25fc16ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:50.988870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c5a5a0-649f-409d-8a34-48f25fc16ac3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:50.989549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5a5a0-649f-409d-8a34-48f25fc16ac3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:14:50.989853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5a5a0-649f-409d-8a34-48f25fc16ac3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:50.991442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c5a5a0-649f-409d-8a34-48f25fc16ac3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:51.005603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1180/2347] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:14:51.010076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ca921-295f-4fd9-95d7-c0bc29f00d79 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:51.019359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ca921-295f-4fd9-95d7-c0bc29f00d79 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:51.024135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0ed99-c812-4650-9fdb-37d5cb608031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:51.024733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0ed99-c812-4650-9fdb-37d5cb608031 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:51.025021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0ed99-c812-4650-9fdb-37d5cb608031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:51.025314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0ed99-c812-4650-9fdb-37d5cb608031 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:51.025943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0ed99-c812-4650-9fdb-37d5cb608031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:51.027825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a0ed99-c812-4650-9fdb-37d5cb608031 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:51.056881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1168/2348] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:14:51 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:14:51.062649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5a5a0-649f-409d-8a34-48f25fc16ac3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:51.079556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c5a5a0-649f-409d-8a34-48f25fc16ac3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:51.150572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aadd6-0d81-46f5-a037-b6c10f0c9d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:51.151187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aadd6-0d81-46f5-a037-b6c10f0c9d3b admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:51.151460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aadd6-0d81-46f5-a037-b6c10f0c9d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:51.151839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2aadd6-0d81-46f5-a037-b6c10f0c9d3b admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:51.152597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2aadd6-0d81-46f5-a037-b6c10f0c9d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:51.154719 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2aadd6-0d81-46f5-a037-b6c10f0c9d3b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:51.176273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1181/2349] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:14:51 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:14:51.180654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0ed99-c812-4650-9fdb-37d5cb608031 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:51.191848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0ed99-c812-4650-9fdb-37d5cb608031 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:51.196572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ce5fb-1539-4675-b5e3-051bda1170a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:51.197197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ce5fb-1539-4675-b5e3-051bda1170a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:51.197478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ce5fb-1539-4675-b5e3-051bda1170a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:51.197750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ce5fb-1539-4675-b5e3-051bda1170a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:51.198388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ce5fb-1539-4675-b5e3-051bda1170a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:51.200201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649ce5fb-1539-4675-b5e3-051bda1170a8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:51.209961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1169/2350] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:14:51.213875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aadd6-0d81-46f5-a037-b6c10f0c9d3b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:51.223724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2aadd6-0d81-46f5-a037-b6c10f0c9d3b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:51.228274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e806b7-62ad-4ac1-a6c5-f1b82969ce34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:51.228858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e806b7-62ad-4ac1-a6c5-f1b82969ce34 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:51.229165 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e806b7-62ad-4ac1-a6c5-f1b82969ce34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:51.229441 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e806b7-62ad-4ac1-a6c5-f1b82969ce34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:51.230078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e806b7-62ad-4ac1-a6c5-f1b82969ce34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:51.232171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e806b7-62ad-4ac1-a6c5-f1b82969ce34 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:51.242851 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1182/2351] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:14:51.247403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ce5fb-1539-4675-b5e3-051bda1170a8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:51.257419 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649ce5fb-1539-4675-b5e3-051bda1170a8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:51.262294 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cf284-c5c0-4195-a513-cb6a765c4b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:51.262931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cf284-c5c0-4195-a513-cb6a765c4b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:51.263205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cf284-c5c0-4195-a513-cb6a765c4b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:51.263496 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0cf284-c5c0-4195-a513-cb6a765c4b7c admin admin] PATH_INFO: `/v3/projects/e9c311fa04f241249d428df8e88539f4/users/cc89baeb82444da78312c4b0b3fb0c41/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:51.278623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cf284-c5c0-4195-a513-cb6a765c4b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9c311fa04f241249d428df8e88539f4, user_id=cc89baeb82444da78312c4b0b3fb0c41, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:51.281275 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0cf284-c5c0-4195-a513-cb6a765c4b7c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:51.283684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:14:51.296948 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1170/2352] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:14:51 2022] PUT /identity/v3/projects/e9c311fa04f241249d428df8e88539f4/users/cc89baeb82444da78312c4b0b3fb0c41/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:14:51.301890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23079c74-462b-4fcd-8315-be1c7d941fae None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:51.302176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23079c74-462b-4fcd-8315-be1c7d941fae None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:51.302447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23079c74-462b-4fcd-8315-be1c7d941fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:51.320929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-23079c74-462b-4fcd-8315-be1c7d941fae None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:51.322114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-23079c74-462b-4fcd-8315-be1c7d941fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:51.353259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:14:51.385237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1183/2353] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:14:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:51.394770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cf284-c5c0-4195-a513-cb6a765c4b7c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:51.396543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5f0cf284-c5c0-4195-a513-cb6a765c4b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:51.453336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0cf284-c5c0-4195-a513-cb6a765c4b7c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:51.456971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affff04-116a-41f9-957c-e08a2ef2553b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:51.457603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affff04-116a-41f9-957c-e08a2ef2553b service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:51.457989 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affff04-116a-41f9-957c-e08a2ef2553b service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:51.458263 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affff04-116a-41f9-957c-e08a2ef2553b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:51.467683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affff04-116a-41f9-957c-e08a2ef2553b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:51.469245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affff04-116a-41f9-957c-e08a2ef2553b service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:51.478406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1171/2354] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:14:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:14:55.329689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90674a65-7e34-437f-aef4-36b58443b274 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:55.330166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90674a65-7e34-437f-aef4-36b58443b274 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:55.330166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90674a65-7e34-437f-aef4-36b58443b274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:55.361171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-90674a65-7e34-437f-aef4-36b58443b274 None None] MFA Rules not processed for user `cc89baeb82444da78312c4b0b3fb0c41`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:14:55.362391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-90674a65-7e34-437f-aef4-36b58443b274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:14:55.431456 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1184/2355] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:14:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:14:55.436494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affff04-116a-41f9-957c-e08a2ef2553b service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:55.448081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affff04-116a-41f9-957c-e08a2ef2553b service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:55.453478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a06f2-e5d8-4149-99d0-2baf663f8877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:55.454085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a06f2-e5d8-4149-99d0-2baf663f8877 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:55.454394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a06f2-e5d8-4149-99d0-2baf663f8877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:55.454666 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a06f2-e5d8-4149-99d0-2baf663f8877 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:55.455443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a06f2-e5d8-4149-99d0-2baf663f8877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:55.457413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a06f2-e5d8-4149-99d0-2baf663f8877 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:55.490319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1172/2356] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:14:55 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:14:55.494434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90674a65-7e34-437f-aef4-36b58443b274 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:55.516866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90674a65-7e34-437f-aef4-36b58443b274 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:55.602078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37c077-f3c7-48c7-a3d6-6a6d9b592e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:55.602695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c077-f3c7-48c7-a3d6-6a6d9b592e97 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:55.602976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c077-f3c7-48c7-a3d6-6a6d9b592e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:55.603251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce37c077-f3c7-48c7-a3d6-6a6d9b592e97 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:55.603894 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37c077-f3c7-48c7-a3d6-6a6d9b592e97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:55.606334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce37c077-f3c7-48c7-a3d6-6a6d9b592e97 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:55.634031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1185/2357] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:14:55 2022] POST /identity/v3/users => generated 294 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:14:55.635710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a06f2-e5d8-4149-99d0-2baf663f8877 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:55.649387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a06f2-e5d8-4149-99d0-2baf663f8877 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:55.655249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff23610-e864-45f9-8a10-d6a6008f900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:55.655961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23610-e864-45f9-8a10-d6a6008f900e admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:55.656136 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23610-e864-45f9-8a10-d6a6008f900e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:55.656409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff23610-e864-45f9-8a10-d6a6008f900e admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:55.657053 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff23610-e864-45f9-8a10-d6a6008f900e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:55.658807 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff23610-e864-45f9-8a10-d6a6008f900e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:55.674489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1173/2358] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:14:55.678560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37c077-f3c7-48c7-a3d6-6a6d9b592e97 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:55.689377 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce37c077-f3c7-48c7-a3d6-6a6d9b592e97 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:55.694148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583be9b-107c-489f-a079-13576ed20e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:55.694837 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583be9b-107c-489f-a079-13576ed20e6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:55.695112 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583be9b-107c-489f-a079-13576ed20e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:55.695582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3583be9b-107c-489f-a079-13576ed20e6e admin admin] PATH_INFO: `/v3/projects/316ac566e6c247c899de1498fdd81206/users/1c20d023d8fc4b31b7f306ee58e3bada/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:55.712429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583be9b-107c-489f-a079-13576ed20e6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=316ac566e6c247c899de1498fdd81206, user_id=1c20d023d8fc4b31b7f306ee58e3bada, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:55.720624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3583be9b-107c-489f-a079-13576ed20e6e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:55.738988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1186/2359] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:14:55 2022] PUT /identity/v3/projects/316ac566e6c247c899de1498fdd81206/users/1c20d023d8fc4b31b7f306ee58e3bada/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:14:55.742905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff23610-e864-45f9-8a10-d6a6008f900e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:55.754095 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff23610-e864-45f9-8a10-d6a6008f900e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:55.826757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6f312e-f437-4edd-b60f-7707b18472bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:55.827475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f312e-f437-4edd-b60f-7707b18472bd admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:55.827824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f312e-f437-4edd-b60f-7707b18472bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:55.828170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6f312e-f437-4edd-b60f-7707b18472bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:55.828912 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f312e-f437-4edd-b60f-7707b18472bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:55.830649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6f312e-f437-4edd-b60f-7707b18472bd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:55.844448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1174/2360] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:14:55.848467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583be9b-107c-489f-a079-13576ed20e6e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:14:55.860173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3583be9b-107c-489f-a079-13576ed20e6e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:14:55.865411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf74b1-c28c-41b6-ac8c-2c0a41642f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:14:55.866176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf74b1-c28c-41b6-ac8c-2c0a41642f87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:14:55.866572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf74b1-c28c-41b6-ac8c-2c0a41642f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:14:55.866919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caf74b1-c28c-41b6-ac8c-2c0a41642f87 admin admin] PATH_INFO: `/v3/projects/316ac566e6c247c899de1498fdd81206/users/1c20d023d8fc4b31b7f306ee58e3bada/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:14:55.869217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf74b1-c28c-41b6-ac8c-2c0a41642f87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=316ac566e6c247c899de1498fdd81206, user_id=1c20d023d8fc4b31b7f306ee58e3bada, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:14:55.876605 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caf74b1-c28c-41b6-ac8c-2c0a41642f87 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:14:55.889819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1187/2361] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:14:55 2022] PUT /identity/v3/projects/316ac566e6c247c899de1498fdd81206/users/1c20d023d8fc4b31b7f306ee58e3bada/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:15:00.734598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707d166-22b2-444d-b200-4491db853f5c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:00.734598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707d166-22b2-444d-b200-4491db853f5c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:00.735191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a707d166-22b2-444d-b200-4491db853f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:00.775181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a707d166-22b2-444d-b200-4491db853f5c None None] MFA Rules not processed for user `1c20d023d8fc4b31b7f306ee58e3bada`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:00.776718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a707d166-22b2-444d-b200-4491db853f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:00.862559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:00.897453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1175/2362] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:15:00.927819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf74b1-c28c-41b6-ac8c-2c0a41642f87 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:00.930972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0caf74b1-c28c-41b6-ac8c-2c0a41642f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:00.972630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:01.144888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caf74b1-c28c-41b6-ac8c-2c0a41642f87 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:01.159141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c15f0d-15d6-42fc-a2d0-80c7c11dfc36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:01.160042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c15f0d-15d6-42fc-a2d0-80c7c11dfc36 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:01.160542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c15f0d-15d6-42fc-a2d0-80c7c11dfc36 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:01.161013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c15f0d-15d6-42fc-a2d0-80c7c11dfc36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:01.239893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c15f0d-15d6-42fc-a2d0-80c7c11dfc36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:01.242227 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c15f0d-15d6-42fc-a2d0-80c7c11dfc36 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:01.258689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1188/2363] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:15:10.893225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=7 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:11.206854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580ffeb-226c-4c0d-a6fa-555a57a2db4a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:11.207068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580ffeb-226c-4c0d-a6fa-555a57a2db4a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:11.207369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b580ffeb-226c-4c0d-a6fa-555a57a2db4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:11.230854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b580ffeb-226c-4c0d-a6fa-555a57a2db4a None None] MFA Rules not processed for user `bcb617f1f41c467fba9717f7f825ac80`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:11.232670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b580ffeb-226c-4c0d-a6fa-555a57a2db4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:11.252226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=48 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:11.299620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1176/2364] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:15:11.951465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c15f0d-15d6-42fc-a2d0-80c7c11dfc36 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:11.952891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-32c15f0d-15d6-42fc-a2d0-80c7c11dfc36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:12.013264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c15f0d-15d6-42fc-a2d0-80c7c11dfc36 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:12.018080 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d8d96-c8cc-4c13-98b8-f5b64a01eac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:12.018698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d8d96-c8cc-4c13-98b8-f5b64a01eac1 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:12.018981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d8d96-c8cc-4c13-98b8-f5b64a01eac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:12.019258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d8d96-c8cc-4c13-98b8-f5b64a01eac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:12.021071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-113d8d96-c8cc-4c13-98b8-f5b64a01eac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:12.036181 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d8d96-c8cc-4c13-98b8-f5b64a01eac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:12.037818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d8d96-c8cc-4c13-98b8-f5b64a01eac1 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:12.051405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1189/2365] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:15:13.180817 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580ffeb-226c-4c0d-a6fa-555a57a2db4a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:13.190609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b580ffeb-226c-4c0d-a6fa-555a57a2db4a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:13.194265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c502505-d284-4ebb-9f4a-8578b3a6a79c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:13.194889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c502505-d284-4ebb-9f4a-8578b3a6a79c service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:13.195168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c502505-d284-4ebb-9f4a-8578b3a6a79c service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:13.195440 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c502505-d284-4ebb-9f4a-8578b3a6a79c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:13.197174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0c502505-d284-4ebb-9f4a-8578b3a6a79c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:13.212509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c502505-d284-4ebb-9f4a-8578b3a6a79c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:13.217255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c502505-d284-4ebb-9f4a-8578b3a6a79c service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:13.228001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1177/2366] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:15:18.673089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d8d96-c8cc-4c13-98b8-f5b64a01eac1 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:18.675448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-113d8d96-c8cc-4c13-98b8-f5b64a01eac1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:18.767465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d8d96-c8cc-4c13-98b8-f5b64a01eac1 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:18.772006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523e3774-e0a3-40aa-9276-b6bbdec42685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:18.772648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e3774-e0a3-40aa-9276-b6bbdec42685 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:18.772921 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e3774-e0a3-40aa-9276-b6bbdec42685 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:18.773291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523e3774-e0a3-40aa-9276-b6bbdec42685 admin admin] PATH_INFO: `/v3/users/cd913a3c0cff4787a90603f3afb2e883` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:18.774455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e3774-e0a3-40aa-9276-b6bbdec42685 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd913a3c0cff4787a90603f3afb2e883)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:18.776637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523e3774-e0a3-40aa-9276-b6bbdec42685 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:18.826880 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-523e3774-e0a3-40aa-9276-b6bbdec42685 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd913a3c0cff4787a90603f3afb2e883'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:18.827640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-523e3774-e0a3-40aa-9276-b6bbdec42685 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd913a3c0cff4787a90603f3afb2e883'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:18.828307 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-523e3774-e0a3-40aa-9276-b6bbdec42685 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd913a3c0cff4787a90603f3afb2e883'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:18.842542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-523e3774-e0a3-40aa-9276-b6bbdec42685 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd913a3c0cff4787a90603f3afb2e883'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:18.849978 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-523e3774-e0a3-40aa-9276-b6bbdec42685 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd913a3c0cff4787a90603f3afb2e883'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:18.862895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1190/2367] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:15:18 2022] DELETE /identity/v3/users/cd913a3c0cff4787a90603f3afb2e883 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:15:19.037733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c502505-d284-4ebb-9f4a-8578b3a6a79c service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:19.052765 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c502505-d284-4ebb-9f4a-8578b3a6a79c service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:19.110822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:19.111454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:19.111732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:19.112009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] PATH_INFO: `/v3/projects/d58361c0e40c48f4966aee3515859326` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:19.113374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d58361c0e40c48f4966aee3515859326)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:19.115295 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:19.164670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd58361c0e40c48f4966aee3515859326', 'request_id': 'req-79802e14-04c7-4d03-8917-37771bab32bd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:19.173491 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd58361c0e40c48f4966aee3515859326', 'request_id': 'req-79802e14-04c7-4d03-8917-37771bab32bd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:19.180442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd58361c0e40c48f4966aee3515859326', 'request_id': 'req-79802e14-04c7-4d03-8917-37771bab32bd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:19.205200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] The token cache is being invalidate because project d58361c0e40c48f4966aee3515859326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:15:19.205513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:19.206512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-79802e14-04c7-4d03-8917-37771bab32bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:19.213673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1178/2368] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:15:19 2022] DELETE /identity/v3/projects/d58361c0e40c48f4966aee3515859326 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:15:19.214909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3531fd-f0cc-4dc7-9717-63fb93676f6e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:19.215271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3531fd-f0cc-4dc7-9717-63fb93676f6e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:19.215617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3531fd-f0cc-4dc7-9717-63fb93676f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:19.245226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7b3531fd-f0cc-4dc7-9717-63fb93676f6e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:19.247689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7b3531fd-f0cc-4dc7-9717-63fb93676f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:19.270389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:19.326585 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1191/2369] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:15:19.328702 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1707d10-baa6-4814-b910-1f06a60b6cf6 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:19.329340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1707d10-baa6-4814-b910-1f06a60b6cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:19.329729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1707d10-baa6-4814-b910-1f06a60b6cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:19.351023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e1707d10-baa6-4814-b910-1f06a60b6cf6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:19.352363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e1707d10-baa6-4814-b910-1f06a60b6cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:19.363972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1179/2370] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:15:19.368175 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3531fd-f0cc-4dc7-9717-63fb93676f6e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:19.380229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3531fd-f0cc-4dc7-9717-63fb93676f6e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:19.385640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd324933-717d-42bf-9082-5fa9e11a8888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:19.386358 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324933-717d-42bf-9082-5fa9e11a8888 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:19.386641 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324933-717d-42bf-9082-5fa9e11a8888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:19.386966 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd324933-717d-42bf-9082-5fa9e11a8888 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:19.387923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd324933-717d-42bf-9082-5fa9e11a8888 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:15:19.388233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd324933-717d-42bf-9082-5fa9e11a8888 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:19.389928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd324933-717d-42bf-9082-5fa9e11a8888 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:19.403756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1192/2371] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:15:19.408447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1707d10-baa6-4814-b910-1f06a60b6cf6 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:19.419103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1707d10-baa6-4814-b910-1f06a60b6cf6 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:19.424672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4bbb3-49c6-49c4-ae9e-9569849a0d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:19.425445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4bbb3-49c6-49c4-ae9e-9569849a0d9a admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:19.425580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4bbb3-49c6-49c4-ae9e-9569849a0d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:19.426144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4bbb3-49c6-49c4-ae9e-9569849a0d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:19.426791 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4bbb3-49c6-49c4-ae9e-9569849a0d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:19.429581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4bbb3-49c6-49c4-ae9e-9569849a0d9a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:19.476438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:19.496080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1180/2372] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:15:19 2022] POST /identity/v3/projects => generated 391 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:15:19.499856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd324933-717d-42bf-9082-5fa9e11a8888 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:19.510346 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd324933-717d-42bf-9082-5fa9e11a8888 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:19.573007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b804fdc-ba72-421d-9c5a-249f3f4b090f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:19.573692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b804fdc-ba72-421d-9c5a-249f3f4b090f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:19.573975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b804fdc-ba72-421d-9c5a-249f3f4b090f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:19.574244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b804fdc-ba72-421d-9c5a-249f3f4b090f admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:19.574879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b804fdc-ba72-421d-9c5a-249f3f4b090f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:19.576759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b804fdc-ba72-421d-9c5a-249f3f4b090f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:19.597796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1193/2373] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:15:19 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:15:19.601962 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4bbb3-49c6-49c4-ae9e-9569849a0d9a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:19.613089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4bbb3-49c6-49c4-ae9e-9569849a0d9a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:19.617800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bfd56-379e-4104-8d06-a00c140e796e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:19.618388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bfd56-379e-4104-8d06-a00c140e796e admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:19.618662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bfd56-379e-4104-8d06-a00c140e796e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:19.618955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bfd56-379e-4104-8d06-a00c140e796e admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:19.619627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bfd56-379e-4104-8d06-a00c140e796e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:19.621235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bfd56-379e-4104-8d06-a00c140e796e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:19.631044 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1181/2374] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:15:19.635417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b804fdc-ba72-421d-9c5a-249f3f4b090f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:19.645239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b804fdc-ba72-421d-9c5a-249f3f4b090f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:19.649740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf50eb9-f457-4787-aa9c-8b06c3ae5cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:19.650899 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf50eb9-f457-4787-aa9c-8b06c3ae5cfd admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:19.651173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf50eb9-f457-4787-aa9c-8b06c3ae5cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:19.651436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf50eb9-f457-4787-aa9c-8b06c3ae5cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:19.652065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf50eb9-f457-4787-aa9c-8b06c3ae5cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:19.653679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf50eb9-f457-4787-aa9c-8b06c3ae5cfd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:19.664007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1194/2375] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:15:19.668626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bfd56-379e-4104-8d06-a00c140e796e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:19.678697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bfd56-379e-4104-8d06-a00c140e796e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:19.683464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3546459-fc92-4232-ab29-20e6539f81a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:19.684125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3546459-fc92-4232-ab29-20e6539f81a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:19.684363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3546459-fc92-4232-ab29-20e6539f81a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:19.684637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3546459-fc92-4232-ab29-20e6539f81a5 admin admin] PATH_INFO: `/v3/projects/32bbc148c9be4ce38aeae7a4e5b9542b/users/0c8b1d97d8194dff9fa78a00a1ec4867/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:19.698694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3546459-fc92-4232-ab29-20e6539f81a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32bbc148c9be4ce38aeae7a4e5b9542b, user_id=0c8b1d97d8194dff9fa78a00a1ec4867, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:19.701346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3546459-fc92-4232-ab29-20e6539f81a5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:19.714980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1182/2376] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:15:19 2022] PUT /identity/v3/projects/32bbc148c9be4ce38aeae7a4e5b9542b/users/0c8b1d97d8194dff9fa78a00a1ec4867/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:15:19.725986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9342926-6122-48e2-a293-04ead2c4a5b3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:19.726503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9342926-6122-48e2-a293-04ead2c4a5b3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:19.726603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9342926-6122-48e2-a293-04ead2c4a5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:19.757745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b9342926-6122-48e2-a293-04ead2c4a5b3 None None] MFA Rules not processed for user `0c8b1d97d8194dff9fa78a00a1ec4867`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:19.758976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b9342926-6122-48e2-a293-04ead2c4a5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:19.825608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1195/2377] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:15:19.836131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3546459-fc92-4232-ab29-20e6539f81a5 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:19.838602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f3546459-fc92-4232-ab29-20e6539f81a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:19.894858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3546459-fc92-4232-ab29-20e6539f81a5 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:19.898496 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fda8a-d475-45c6-837b-faf6fd217850 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:19.899083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fda8a-d475-45c6-837b-faf6fd217850 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:19.899360 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fda8a-d475-45c6-837b-faf6fd217850 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:19.899663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68fda8a-d475-45c6-837b-faf6fd217850 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:19.908530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fda8a-d475-45c6-837b-faf6fd217850 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:19.909975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68fda8a-d475-45c6-837b-faf6fd217850 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:19.919164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1183/2378] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:19 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:15:20.474256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9342926-6122-48e2-a293-04ead2c4a5b3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:20.475599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b9342926-6122-48e2-a293-04ead2c4a5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:20.496317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:20.550461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9342926-6122-48e2-a293-04ead2c4a5b3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:20.555848 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab87c85-f1c9-42f4-b7e9-657db67b40ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:20.556463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab87c85-f1c9-42f4-b7e9-657db67b40ab service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:20.556863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab87c85-f1c9-42f4-b7e9-657db67b40ab service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:20.557169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab87c85-f1c9-42f4-b7e9-657db67b40ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:20.558995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9ab87c85-f1c9-42f4-b7e9-657db67b40ab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:20.618335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab87c85-f1c9-42f4-b7e9-657db67b40ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:20.619952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab87c85-f1c9-42f4-b7e9-657db67b40ab service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:20.628997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1196/2379] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:15:29.914632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:INSERT=2,keystone:SELECT=43 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:30.624596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=48 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:39.438626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fda8a-d475-45c6-837b-faf6fd217850 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:39.440149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d68fda8a-d475-45c6-837b-faf6fd217850 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:39.496723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68fda8a-d475-45c6-837b-faf6fd217850 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:39.500267 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71237fe-5ddc-45d9-90df-1f811c6e617e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:39.500857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71237fe-5ddc-45d9-90df-1f811c6e617e service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:39.501172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71237fe-5ddc-45d9-90df-1f811c6e617e service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:39.501450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71237fe-5ddc-45d9-90df-1f811c6e617e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:39.503137 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a71237fe-5ddc-45d9-90df-1f811c6e617e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:39.512431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71237fe-5ddc-45d9-90df-1f811c6e617e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:39.513870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71237fe-5ddc-45d9-90df-1f811c6e617e service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:39.521870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1184/2380] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:15:43.054160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655e806-e9e5-4653-a79a-f791c70964cb None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:43.054723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655e806-e9e5-4653-a79a-f791c70964cb None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:43.054826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7655e806-e9e5-4653-a79a-f791c70964cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:43.075059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7655e806-e9e5-4653-a79a-f791c70964cb None None] MFA Rules not processed for user `cc89baeb82444da78312c4b0b3fb0c41`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:43.076279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7655e806-e9e5-4653-a79a-f791c70964cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:43.131334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1197/2381] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:15:43.476951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71237fe-5ddc-45d9-90df-1f811c6e617e service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:43.485721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71237fe-5ddc-45d9-90df-1f811c6e617e service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:43.489333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bf90b-6dc5-4ddb-8148-281390c9f686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:43.489928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bf90b-6dc5-4ddb-8148-281390c9f686 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:43.490231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bf90b-6dc5-4ddb-8148-281390c9f686 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:43.490497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6bf90b-6dc5-4ddb-8148-281390c9f686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:43.492732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8d6bf90b-6dc5-4ddb-8148-281390c9f686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:43.502343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6bf90b-6dc5-4ddb-8148-281390c9f686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:43.503774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6bf90b-6dc5-4ddb-8148-281390c9f686 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:43.512396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1185/2382] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:15:43.964943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655e806-e9e5-4653-a79a-f791c70964cb None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:43.966324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7655e806-e9e5-4653-a79a-f791c70964cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:44.037525 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7655e806-e9e5-4653-a79a-f791c70964cb None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:44.043155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b9103-b4e5-48f4-b895-c8cfda37a8ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:44.044013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b9103-b4e5-48f4-b895-c8cfda37a8ad service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:44.044286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b9103-b4e5-48f4-b895-c8cfda37a8ad service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:44.044553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b9103-b4e5-48f4-b895-c8cfda37a8ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:44.046281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-274b9103-b4e5-48f4-b895-c8cfda37a8ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:44.058129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b9103-b4e5-48f4-b895-c8cfda37a8ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:44.059706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b9103-b4e5-48f4-b895-c8cfda37a8ad service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:44.069613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1198/2383] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:15:46.608578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bf90b-6dc5-4ddb-8148-281390c9f686 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:46.674508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6bf90b-6dc5-4ddb-8148-281390c9f686 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:46.679045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:46.679667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:46.679948 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:46.680857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] PATH_INFO: `/v3/users/0c8b1d97d8194dff9fa78a00a1ec4867` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:46.682066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c8b1d97d8194dff9fa78a00a1ec4867)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:46.683975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:46.735023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c8b1d97d8194dff9fa78a00a1ec4867'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:46.735840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c8b1d97d8194dff9fa78a00a1ec4867'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:46.744996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c8b1d97d8194dff9fa78a00a1ec4867'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:46.745793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c8b1d97d8194dff9fa78a00a1ec4867'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:46.761122 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c8b1d97d8194dff9fa78a00a1ec4867'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:46.774392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1186/2384] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:15:46 2022] DELETE /identity/v3/users/0c8b1d97d8194dff9fa78a00a1ec4867 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:15:46.778837 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87345b7a-6c1b-4775-bab3-8159b1e8b651 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:46.779320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87345b7a-6c1b-4775-bab3-8159b1e8b651 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:46.779483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87345b7a-6c1b-4775-bab3-8159b1e8b651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:46.799257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-87345b7a-6c1b-4775-bab3-8159b1e8b651 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:46.800479 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-87345b7a-6c1b-4775-bab3-8159b1e8b651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:46.874532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1199/2385] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:15:46.890729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:46.893493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:46.906125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:46.922950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87345b7a-6c1b-4775-bab3-8159b1e8b651 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:46.924478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-87345b7a-6c1b-4775-bab3-8159b1e8b651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:46.965682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f639f-1a5d-4c6e-ac74-5fe35eb44750 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:46.970178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a3b87-0370-4be8-bc64-6ac1e2debb21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:46.970799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a3b87-0370-4be8-bc64-6ac1e2debb21 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:46.971106 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a3b87-0370-4be8-bc64-6ac1e2debb21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:46.971388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a3b87-0370-4be8-bc64-6ac1e2debb21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:46.980359 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:46.982952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a3b87-0370-4be8-bc64-6ac1e2debb21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:46.984609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a3b87-0370-4be8-bc64-6ac1e2debb21 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:46.997160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1187/2386] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:15:47.022145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87345b7a-6c1b-4775-bab3-8159b1e8b651 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:47.029254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23167ce4-fa6f-415e-a81d-f9e3a278eb4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:47.030165 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23167ce4-fa6f-415e-a81d-f9e3a278eb4d service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:47.030636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23167ce4-fa6f-415e-a81d-f9e3a278eb4d service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:47.031096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23167ce4-fa6f-415e-a81d-f9e3a278eb4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:47.033520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-23167ce4-fa6f-415e-a81d-f9e3a278eb4d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:47.049507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23167ce4-fa6f-415e-a81d-f9e3a278eb4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:47.051119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23167ce4-fa6f-415e-a81d-f9e3a278eb4d service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:47.063797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1200/2387] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:15:47.439871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a3b87-0370-4be8-bc64-6ac1e2debb21 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:47.443581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-395a3b87-0370-4be8-bc64-6ac1e2debb21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:47.458707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a3b87-0370-4be8-bc64-6ac1e2debb21 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:47.464571 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:47.465258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:47.465542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:47.465812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] PATH_INFO: `/v3/projects/32bbc148c9be4ce38aeae7a4e5b9542b` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:47.466959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32bbc148c9be4ce38aeae7a4e5b9542b)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:47.468898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:47.523750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32bbc148c9be4ce38aeae7a4e5b9542b', 'request_id': 'req-31390d7c-4a6e-4ff0-9fe7-175a49e56999'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:47.532675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32bbc148c9be4ce38aeae7a4e5b9542b', 'request_id': 'req-31390d7c-4a6e-4ff0-9fe7-175a49e56999'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:47.541604 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32bbc148c9be4ce38aeae7a4e5b9542b', 'request_id': 'req-31390d7c-4a6e-4ff0-9fe7-175a49e56999'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:47.554949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] The token cache is being invalidate because project 32bbc148c9be4ce38aeae7a4e5b9542b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:15:47.555870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:47.556836 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31390d7c-4a6e-4ff0-9fe7-175a49e56999 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:15:47.558800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1188/2388] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:15:47 2022] DELETE /identity/v3/projects/32bbc148c9be4ce38aeae7a4e5b9542b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:15:47.563439 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63864ba0-ac9b-483a-9758-75b8cf4d520a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:47.563713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63864ba0-ac9b-483a-9758-75b8cf4d520a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:47.564022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63864ba0-ac9b-483a-9758-75b8cf4d520a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:47.583711 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-63864ba0-ac9b-483a-9758-75b8cf4d520a None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:47.584919 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-63864ba0-ac9b-483a-9758-75b8cf4d520a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:47.649167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1201/2389] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:15:47.657173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc5892-52b9-4b90-8e93-4e19995428c1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:47.657641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc5892-52b9-4b90-8e93-4e19995428c1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:47.657739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc5892-52b9-4b90-8e93-4e19995428c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:47.678203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a2cc5892-52b9-4b90-8e93-4e19995428c1 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:47.679419 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a2cc5892-52b9-4b90-8e93-4e19995428c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:47.687818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1189/2390] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:15:47.691930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63864ba0-ac9b-483a-9758-75b8cf4d520a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:47.701494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63864ba0-ac9b-483a-9758-75b8cf4d520a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:47.706030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8434a6-1423-4409-ac3f-75bd0dbcd10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:47.706625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8434a6-1423-4409-ac3f-75bd0dbcd10d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:47.706900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8434a6-1423-4409-ac3f-75bd0dbcd10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:47.707169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8434a6-1423-4409-ac3f-75bd0dbcd10d admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:47.707808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8434a6-1423-4409-ac3f-75bd0dbcd10d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:15:47.708109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8434a6-1423-4409-ac3f-75bd0dbcd10d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:47.709726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8434a6-1423-4409-ac3f-75bd0dbcd10d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:47.721400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1202/2391] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:15:47.725515 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc5892-52b9-4b90-8e93-4e19995428c1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:47.735380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc5892-52b9-4b90-8e93-4e19995428c1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:47.739976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e34b7-9a6f-40da-ab27-937dc7753f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:47.740554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e34b7-9a6f-40da-ab27-937dc7753f82 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:47.740829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e34b7-9a6f-40da-ab27-937dc7753f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:47.741092 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e34b7-9a6f-40da-ab27-937dc7753f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:47.741740 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e34b7-9a6f-40da-ab27-937dc7753f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:47.744245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e34b7-9a6f-40da-ab27-937dc7753f82 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:47.771563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1190/2392] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:15:47 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:15:47.775817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8434a6-1423-4409-ac3f-75bd0dbcd10d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:47.785877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8434a6-1423-4409-ac3f-75bd0dbcd10d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:47.843170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427fff9-1096-44b5-9cda-cc147fa6e646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:47.843771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427fff9-1096-44b5-9cda-cc147fa6e646 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:47.844070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427fff9-1096-44b5-9cda-cc147fa6e646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:47.844320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427fff9-1096-44b5-9cda-cc147fa6e646 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:47.844951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427fff9-1096-44b5-9cda-cc147fa6e646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:47.846885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427fff9-1096-44b5-9cda-cc147fa6e646 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:47.869390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1203/2393] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:15:47 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:15:47.873455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e34b7-9a6f-40da-ab27-937dc7753f82 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:47.883565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e34b7-9a6f-40da-ab27-937dc7753f82 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:47.888163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b493406-5913-42a0-9c26-f8240b4338f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:47.888742 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b493406-5913-42a0-9c26-f8240b4338f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:47.889014 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b493406-5913-42a0-9c26-f8240b4338f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:47.889318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b493406-5913-42a0-9c26-f8240b4338f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:47.889991 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b493406-5913-42a0-9c26-f8240b4338f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:47.891548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b493406-5913-42a0-9c26-f8240b4338f8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:47.901847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1191/2394] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:15:47.905674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427fff9-1096-44b5-9cda-cc147fa6e646 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:47.916386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427fff9-1096-44b5-9cda-cc147fa6e646 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:47.921247 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b6c13-6429-4d43-9686-85fdd23be46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:47.921836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b6c13-6429-4d43-9686-85fdd23be46c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:47.922117 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b6c13-6429-4d43-9686-85fdd23be46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:47.922392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b6c13-6429-4d43-9686-85fdd23be46c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:47.923029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b6c13-6429-4d43-9686-85fdd23be46c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:47.924631 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b6c13-6429-4d43-9686-85fdd23be46c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:47.935264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1204/2395] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:15:47.939164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b493406-5913-42a0-9c26-f8240b4338f8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:47.949337 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b493406-5913-42a0-9c26-f8240b4338f8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:47.953889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d92592-98b3-4d28-a439-c4f6bc58a711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:47.954522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92592-98b3-4d28-a439-c4f6bc58a711 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:47.954796 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92592-98b3-4d28-a439-c4f6bc58a711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:47.955067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d92592-98b3-4d28-a439-c4f6bc58a711 admin admin] PATH_INFO: `/v3/projects/53f244703dfd4cb29d183025b9d6789e/users/9e819ca5b7e446ff86dc6a47abd88105/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:47.970097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d92592-98b3-4d28-a439-c4f6bc58a711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f244703dfd4cb29d183025b9d6789e, user_id=9e819ca5b7e446ff86dc6a47abd88105, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:47.972743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d92592-98b3-4d28-a439-c4f6bc58a711 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:47.986296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1192/2396] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:15:47 2022] PUT /identity/v3/projects/53f244703dfd4cb29d183025b9d6789e/users/9e819ca5b7e446ff86dc6a47abd88105/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:15:47.991230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703edac0-a650-4e3d-9599-32ec2cc8b68c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:47.991516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703edac0-a650-4e3d-9599-32ec2cc8b68c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:47.991793 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703edac0-a650-4e3d-9599-32ec2cc8b68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:48.010630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-703edac0-a650-4e3d-9599-32ec2cc8b68c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:48.011838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-703edac0-a650-4e3d-9599-32ec2cc8b68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:48.036878 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:48.081659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1205/2397] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:15:48.091727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d92592-98b3-4d28-a439-c4f6bc58a711 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:48.093479 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-95d92592-98b3-4d28-a439-c4f6bc58a711 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:48.128930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:48.153225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d92592-98b3-4d28-a439-c4f6bc58a711 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:48.156868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66630b22-f6f7-467f-a635-ebf59dee6c60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:48.157498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66630b22-f6f7-467f-a635-ebf59dee6c60 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:48.157779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66630b22-f6f7-467f-a635-ebf59dee6c60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:48.158048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66630b22-f6f7-467f-a635-ebf59dee6c60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:48.167717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66630b22-f6f7-467f-a635-ebf59dee6c60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:48.169372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66630b22-f6f7-467f-a635-ebf59dee6c60 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:48.178696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1193/2398] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:15:52.260954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759422d6-d6c5-407b-883e-b917851bd79b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:52.262076 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759422d6-d6c5-407b-883e-b917851bd79b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:52.262532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759422d6-d6c5-407b-883e-b917851bd79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:52.297143 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-759422d6-d6c5-407b-883e-b917851bd79b None None] MFA Rules not processed for user `9e819ca5b7e446ff86dc6a47abd88105`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:52.298357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-759422d6-d6c5-407b-883e-b917851bd79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:52.372768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1206/2399] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:15:52.377438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66630b22-f6f7-467f-a635-ebf59dee6c60 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:52.388982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66630b22-f6f7-467f-a635-ebf59dee6c60 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:52.398241 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcf99b-d94c-4d26-a39c-f4ad99cbbdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:52.399550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcf99b-d94c-4d26-a39c-f4ad99cbbdb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:52.399929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcf99b-d94c-4d26-a39c-f4ad99cbbdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:52.400302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bcf99b-d94c-4d26-a39c-f4ad99cbbdb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:52.401016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bcf99b-d94c-4d26-a39c-f4ad99cbbdb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:52.403026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bcf99b-d94c-4d26-a39c-f4ad99cbbdb4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:52.434343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1194/2400] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:15:52 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:15:52.438084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759422d6-d6c5-407b-883e-b917851bd79b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:52.456832 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759422d6-d6c5-407b-883e-b917851bd79b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:52.530248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec606e5-624b-4ccb-906a-86fda42a6982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:52.530880 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec606e5-624b-4ccb-906a-86fda42a6982 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:52.531162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec606e5-624b-4ccb-906a-86fda42a6982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:52.531436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec606e5-624b-4ccb-906a-86fda42a6982 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:52.532087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec606e5-624b-4ccb-906a-86fda42a6982 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:52.534129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec606e5-624b-4ccb-906a-86fda42a6982 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:52.560313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1207/2401] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:15:52 2022] POST /identity/v3/users => generated 298 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:15:52.564468 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcf99b-d94c-4d26-a39c-f4ad99cbbdb4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:52.579026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bcf99b-d94c-4d26-a39c-f4ad99cbbdb4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:52.583895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256c0cc-a077-4c2a-b0bf-32f6b8733051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:52.584497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256c0cc-a077-4c2a-b0bf-32f6b8733051 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:52.584772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256c0cc-a077-4c2a-b0bf-32f6b8733051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:52.585038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256c0cc-a077-4c2a-b0bf-32f6b8733051 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:52.585726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256c0cc-a077-4c2a-b0bf-32f6b8733051 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:52.587475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256c0cc-a077-4c2a-b0bf-32f6b8733051 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:52.598469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1195/2402] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:15:52.602963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec606e5-624b-4ccb-906a-86fda42a6982 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:52.613076 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec606e5-624b-4ccb-906a-86fda42a6982 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:52.618357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a43a80-8b03-408f-8122-d83ae135959b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:52.619131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a43a80-8b03-408f-8122-d83ae135959b admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:52.619409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a43a80-8b03-408f-8122-d83ae135959b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:52.619684 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a43a80-8b03-408f-8122-d83ae135959b admin admin] PATH_INFO: `/v3/projects/aa2609ad72ba4dc2a4c3566078265457/users/e551791f925b445686da7dee4df54891/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:52.636822 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a43a80-8b03-408f-8122-d83ae135959b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa2609ad72ba4dc2a4c3566078265457, user_id=e551791f925b445686da7dee4df54891, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:52.639729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a43a80-8b03-408f-8122-d83ae135959b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:52.664938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1208/2403] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:15:52 2022] PUT /identity/v3/projects/aa2609ad72ba4dc2a4c3566078265457/users/e551791f925b445686da7dee4df54891/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:15:52.671809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256c0cc-a077-4c2a-b0bf-32f6b8733051 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:52.683157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256c0cc-a077-4c2a-b0bf-32f6b8733051 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:52.746990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582200ff-3a5b-4f1d-9b6f-42f7b734ff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:52.747602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582200ff-3a5b-4f1d-9b6f-42f7b734ff4a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:52.748046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582200ff-3a5b-4f1d-9b6f-42f7b734ff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:52.748326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582200ff-3a5b-4f1d-9b6f-42f7b734ff4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:52.749038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582200ff-3a5b-4f1d-9b6f-42f7b734ff4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:52.750818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582200ff-3a5b-4f1d-9b6f-42f7b734ff4a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:52.762266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1196/2404] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:15:52.766538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a43a80-8b03-408f-8122-d83ae135959b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:52.777228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a43a80-8b03-408f-8122-d83ae135959b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:52.782166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896d731-e373-4b79-8eb8-8196ea0dd75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:52.782837 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896d731-e373-4b79-8eb8-8196ea0dd75b admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:52.783116 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896d731-e373-4b79-8eb8-8196ea0dd75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:52.783390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896d731-e373-4b79-8eb8-8196ea0dd75b admin admin] PATH_INFO: `/v3/projects/aa2609ad72ba4dc2a4c3566078265457/users/e551791f925b445686da7dee4df54891/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:52.785463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896d731-e373-4b79-8eb8-8196ea0dd75b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa2609ad72ba4dc2a4c3566078265457, user_id=e551791f925b445686da7dee4df54891, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:52.788387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896d731-e373-4b79-8eb8-8196ea0dd75b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:52.804188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1209/2405] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:15:52 2022] PUT /identity/v3/projects/aa2609ad72ba4dc2a4c3566078265457/users/e551791f925b445686da7dee4df54891/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:15:57.696845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b24f3-b862-4645-a6ef-ff47d3185847 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:57.697374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b24f3-b862-4645-a6ef-ff47d3185847 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:57.697464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b24f3-b862-4645-a6ef-ff47d3185847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:57.737831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-877b24f3-b862-4645-a6ef-ff47d3185847 None None] MFA Rules not processed for user `e551791f925b445686da7dee4df54891`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:57.738277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-877b24f3-b862-4645-a6ef-ff47d3185847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:57.847353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1197/2406] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:15:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:15:57.860619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896d731-e373-4b79-8eb8-8196ea0dd75b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:57.862484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3896d731-e373-4b79-8eb8-8196ea0dd75b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:57.876211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:57.938425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896d731-e373-4b79-8eb8-8196ea0dd75b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:57.944197 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8eb88-eaf8-40a9-8d95-c4a368902403 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '195565a7695b4d9b82e0ee76181fee2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:57.946371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8eb88-eaf8-40a9-8d95-c4a368902403 service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:57.946371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8eb88-eaf8-40a9-8d95-c4a368902403 service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:57.946371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8eb88-eaf8-40a9-8d95-c4a368902403 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:58.029078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8eb88-eaf8-40a9-8d95-c4a368902403 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:58.030540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8eb88-eaf8-40a9-8d95-c4a368902403 service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:58.041980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1210/2407] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:15:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:15:58.234712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74c7b3-4b5c-474d-aa76-7d0076cbfa81 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:58.235007 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74c7b3-4b5c-474d-aa76-7d0076cbfa81 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:58.235285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74c7b3-4b5c-474d-aa76-7d0076cbfa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:58.259231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ac74c7b3-4b5c-474d-aa76-7d0076cbfa81 None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:15:58.260710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ac74c7b3-4b5c-474d-aa76-7d0076cbfa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:58.282668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:15:58.330934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1198/2408] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:15:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:15:58.402047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8eb88-eaf8-40a9-8d95-c4a368902403 service glance] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:15:58.403814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-64c8eb88-eaf8-40a9-8d95-c4a368902403 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:15:58.493606 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8eb88-eaf8-40a9-8d95-c4a368902403 service glance] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:15:58.498926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06ed99-cd3f-4a91-afcd-81745db48d5a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:15:58.499625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06ed99-cd3f-4a91-afcd-81745db48d5a service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:15:58.499975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06ed99-cd3f-4a91-afcd-81745db48d5a service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:15:58.500321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06ed99-cd3f-4a91-afcd-81745db48d5a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:15:58.510808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06ed99-cd3f-4a91-afcd-81745db48d5a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:15:58.512794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06ed99-cd3f-4a91-afcd-81745db48d5a service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:15:58.522653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1211/2409] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:15:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:16:03.006443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74c7b3-4b5c-474d-aa76-7d0076cbfa81 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:16:03.089834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74c7b3-4b5c-474d-aa76-7d0076cbfa81 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:16:03.096645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c6e47-e271-4851-a086-ce930863226d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:16:03.097841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6e47-e271-4851-a086-ce930863226d service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:16:03.098357 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6e47-e271-4851-a086-ce930863226d service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:16:03.098852 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c6e47-e271-4851-a086-ce930863226d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:16:03.103572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2b1c6e47-e271-4851-a086-ce930863226d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:16:03.194901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c6e47-e271-4851-a086-ce930863226d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:16:03.196063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1c6e47-e271-4851-a086-ce930863226d service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:16:03.209155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1199/2410] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:16:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:16:05.214019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06ed99-cd3f-4a91-afcd-81745db48d5a service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:16:05.216422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7c06ed99-cd3f-4a91-afcd-81745db48d5a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:16:05.302751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:16:05.339183 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06ed99-cd3f-4a91-afcd-81745db48d5a service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:16:05.345071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718717c5-a54e-4d28-936e-4ddc0be211a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:16:05.346230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718717c5-a54e-4d28-936e-4ddc0be211a9 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:16:05.346230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718717c5-a54e-4d28-936e-4ddc0be211a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:16:05.346437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718717c5-a54e-4d28-936e-4ddc0be211a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:16:05.348144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-718717c5-a54e-4d28-936e-4ddc0be211a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:16:05.362840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718717c5-a54e-4d28-936e-4ddc0be211a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:16:05.364485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718717c5-a54e-4d28-936e-4ddc0be211a9 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:16:05.376760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1212/2411] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:16:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:16:08.541227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f2742-43c7-42ef-a9b4-8eef005df227 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:16:08.541831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f2742-43c7-42ef-a9b4-8eef005df227 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:16:08.541831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f2742-43c7-42ef-a9b4-8eef005df227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:16:08.562473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7e7f2742-43c7-42ef-a9b4-8eef005df227 None None] MFA Rules not processed for user `9e819ca5b7e446ff86dc6a47abd88105`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:16:08.563681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7e7f2742-43c7-42ef-a9b4-8eef005df227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:16:08.570593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1200/2412] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:16:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:16:10.824561 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbb91d-f3e8-4e31-be88-8c2f647222df None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:16:10.825098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbb91d-f3e8-4e31-be88-8c2f647222df None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:16:10.825098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbb91d-f3e8-4e31-be88-8c2f647222df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:16:10.844956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-69cbb91d-f3e8-4e31-be88-8c2f647222df None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:16:10.846273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-69cbb91d-f3e8-4e31-be88-8c2f647222df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:16:10.854386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1213/2413] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:16:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:16:10.867319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f2742-43c7-42ef-a9b4-8eef005df227 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:16:10.880255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f2742-43c7-42ef-a9b4-8eef005df227 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:16:10.886509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f93f41-83ed-4b45-a612-272b544865bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:16:10.887120 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f93f41-83ed-4b45-a612-272b544865bd service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:16:10.887402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f93f41-83ed-4b45-a612-272b544865bd service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:16:10.887690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f93f41-83ed-4b45-a612-272b544865bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:16:10.896122 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f93f41-83ed-4b45-a612-272b544865bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:16:10.898241 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f93f41-83ed-4b45-a612-272b544865bd service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:16:10.906417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1201/2414] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:16:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 22:16:15.501638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbb91d-f3e8-4e31-be88-8c2f647222df None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:16:15.522095 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbb91d-f3e8-4e31-be88-8c2f647222df None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:16:15.535346 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0ed98-0cb1-4104-82b1-c4b3c42db9dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:16:15.536092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0ed98-0cb1-4104-82b1-c4b3c42db9dc service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:16:15.536487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0ed98-0cb1-4104-82b1-c4b3c42db9dc service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:16:15.536851 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0ed98-0cb1-4104-82b1-c4b3c42db9dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:16:15.539497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-07d0ed98-0cb1-4104-82b1-c4b3c42db9dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:16:15.604239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0ed98-0cb1-4104-82b1-c4b3c42db9dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:16:15.605873 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0ed98-0cb1-4104-82b1-c4b3c42db9dc service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:16:15.615357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1214/2415] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:16:15 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:16:20.903125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=74 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:16:25.610869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=41 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:16:32.416722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f93f41-83ed-4b45-a612-272b544865bd service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:16:32.418268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-29f93f41-83ed-4b45-a612-272b544865bd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:16:32.436737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f93f41-83ed-4b45-a612-272b544865bd service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:16:32.442255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6e6094-0a52-4f38-a5d5-71093ff05176 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:16:32.442858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6094-0a52-4f38-a5d5-71093ff05176 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:16:32.443129 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6094-0a52-4f38-a5d5-71093ff05176 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:16:32.443414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6e6094-0a52-4f38-a5d5-71093ff05176 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:16:32.445837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-dc6e6094-0a52-4f38-a5d5-71093ff05176 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:16:32.457830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e6094-0a52-4f38-a5d5-71093ff05176 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:16:32.459399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6e6094-0a52-4f38-a5d5-71093ff05176 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:16:32.470290 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1202/2416] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:16:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:16:42.465841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:16:44.836157 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0ed98-0cb1-4104-82b1-c4b3c42db9dc service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:16:44.837672 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-07d0ed98-0cb1-4104-82b1-c4b3c42db9dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:16:44.849707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0ed98-0cb1-4104-82b1-c4b3c42db9dc service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:16:44.854364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fc178-f896-4b2a-9919-8f3aa62a7c3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:16:44.854980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fc178-f896-4b2a-9919-8f3aa62a7c3d service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:16:44.855264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fc178-f896-4b2a-9919-8f3aa62a7c3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:16:44.855535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fc178-f896-4b2a-9919-8f3aa62a7c3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:16:44.865081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fc178-f896-4b2a-9919-8f3aa62a7c3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:16:44.867082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fc178-f896-4b2a-9919-8f3aa62a7c3d service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:16:44.878004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1215/2417] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:16:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:16:54.873245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:17:00.405437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1672dba-de52-41c2-95e3-203bb1b893af None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:17:00.405972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1672dba-de52-41c2-95e3-203bb1b893af None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:17:00.405972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1672dba-de52-41c2-95e3-203bb1b893af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:17:00.425359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b1672dba-de52-41c2-95e3-203bb1b893af None None] MFA Rules not processed for user `ca01fc46c837486b808390ff0c48fa1f`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:17:00.426543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b1672dba-de52-41c2-95e3-203bb1b893af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:17:00.433932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1203/2418] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:17:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:17:10.420675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:17:18.639477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae25a1-1434-448c-a7c0-106f5e35e479 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:17:18.640037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae25a1-1434-448c-a7c0-106f5e35e479 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:17:18.640037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae25a1-1434-448c-a7c0-106f5e35e479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:17:18.659238 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-28ae25a1-1434-448c-a7c0-106f5e35e479 None None] MFA Rules not processed for user `bcb617f1f41c467fba9717f7f825ac80`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:17:18.660507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-28ae25a1-1434-448c-a7c0-106f5e35e479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:17:18.716273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1216/2419] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:17:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:17:19.408226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1672dba-de52-41c2-95e3-203bb1b893af None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:17:19.418793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1672dba-de52-41c2-95e3-203bb1b893af None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:17:19.422375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2337db-0b9f-41a4-a1c5-04511c561c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:17:19.422973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2337db-0b9f-41a4-a1c5-04511c561c7b service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:17:19.423249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2337db-0b9f-41a4-a1c5-04511c561c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:17:19.423519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2337db-0b9f-41a4-a1c5-04511c561c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:17:19.425273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0e2337db-0b9f-41a4-a1c5-04511c561c7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:17:19.434860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2337db-0b9f-41a4-a1c5-04511c561c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:17:19.436287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2337db-0b9f-41a4-a1c5-04511c561c7b service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:17:19.444634 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1204/2420] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:17:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:17:28.708649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=25 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:17:29.440769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:17:39.419102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae25a1-1434-448c-a7c0-106f5e35e479 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:17:39.432526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae25a1-1434-448c-a7c0-106f5e35e479 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:17:39.437863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d00b8-29bc-49c4-b02a-d94498108883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:17:39.438455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d00b8-29bc-49c4-b02a-d94498108883 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:17:39.438728 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d00b8-29bc-49c4-b02a-d94498108883 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:17:39.438997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d00b8-29bc-49c4-b02a-d94498108883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:17:39.440681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8a1d00b8-29bc-49c4-b02a-d94498108883 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:17:39.498517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d00b8-29bc-49c4-b02a-d94498108883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:17:39.500106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d00b8-29bc-49c4-b02a-d94498108883 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:17:39.508999 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1217/2421] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:17:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:17:49.504829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=28 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:18:22.135647 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2337db-0b9f-41a4-a1c5-04511c561c7b service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:22.153947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2337db-0b9f-41a4-a1c5-04511c561c7b service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:22.157585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101407-39e8-4750-ad4b-66726559c6b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:22.158171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101407-39e8-4750-ad4b-66726559c6b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:22.158444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101407-39e8-4750-ad4b-66726559c6b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:22.158711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f101407-39e8-4750-ad4b-66726559c6b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:22.160393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5f101407-39e8-4750-ad4b-66726559c6b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:22.276564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f101407-39e8-4750-ad4b-66726559c6b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:22.278003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f101407-39e8-4750-ad4b-66726559c6b7 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:22.296915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1205/2422] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:18:25.320663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d00b8-29bc-49c4-b02a-d94498108883 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:25.322263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8a1d00b8-29bc-49c4-b02a-d94498108883 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:25.385698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101407-39e8-4750-ad4b-66726559c6b7 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:25.387113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5f101407-39e8-4750-ad4b-66726559c6b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:25.401042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f101407-39e8-4750-ad4b-66726559c6b7 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:25.407751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:25.408679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:25.410009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:25.410474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] PATH_INFO: `/v3/users/ca01fc46c837486b808390ff0c48fa1f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:25.411915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca01fc46c837486b808390ff0c48fa1f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:25.419012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:25.463350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca01fc46c837486b808390ff0c48fa1f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:18:25.463996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca01fc46c837486b808390ff0c48fa1f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:18:25.470986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca01fc46c837486b808390ff0c48fa1f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:18:25.471620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca01fc46c837486b808390ff0c48fa1f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:18:25.484698 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca01fc46c837486b808390ff0c48fa1f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:18:25.494188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:18:25.496590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d00b8-29bc-49c4-b02a-d94498108883 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:25.500234 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1206/2423] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:18:25 2022] DELETE /identity/v3/users/ca01fc46c837486b808390ff0c48fa1f => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:18:25.574130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a244e-5158-4a1f-bd9a-32faebe472da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:25.574680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a244e-5158-4a1f-bd9a-32faebe472da service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:25.574958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a244e-5158-4a1f-bd9a-32faebe472da service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:25.575233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a244e-5158-4a1f-bd9a-32faebe472da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:25.578167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-663a244e-5158-4a1f-bd9a-32faebe472da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:25.593675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a244e-5158-4a1f-bd9a-32faebe472da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:25.595304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a244e-5158-4a1f-bd9a-32faebe472da service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:25.605931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1218/2424] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:18:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:18:25.686732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:25.688201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:25.757719 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6edf8-59bc-4c8e-9e0b-eef7c903db65 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:25.762591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:25.763213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:25.763490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:25.763758 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] PATH_INFO: `/v3/projects/0499a794df4444b594ee3126a8e210c1` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:25.764901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0499a794df4444b594ee3126a8e210c1)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:25.766825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:25.816556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0499a794df4444b594ee3126a8e210c1', 'request_id': 'req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:18:25.826548 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0499a794df4444b594ee3126a8e210c1', 'request_id': 'req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:18:25.833577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0499a794df4444b594ee3126a8e210c1', 'request_id': 'req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:18:25.844644 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] The token cache is being invalidate because project 0499a794df4444b594ee3126a8e210c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:18:25.844955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:18:25.845626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ce64e8e9-817c-4a0d-83e9-64ccc5c10541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:18:25.847291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1207/2425] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:18:25 2022] DELETE /identity/v3/projects/0499a794df4444b594ee3126a8e210c1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:18:25.852691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95b1c7-c00d-41ae-8d3f-25688a572de7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:25.852982 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95b1c7-c00d-41ae-8d3f-25688a572de7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:25.853297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d95b1c7-c00d-41ae-8d3f-25688a572de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:25.887677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:18:25.914719 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-0d95b1c7-c00d-41ae-8d3f-25688a572de7 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:18:25.916037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0d95b1c7-c00d-41ae-8d3f-25688a572de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:25.986565 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1219/2426] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:18:25.995750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d7263-f092-4f8c-8307-a925fe4d0ca7 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:25.996038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d7263-f092-4f8c-8307-a925fe4d0ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:25.996312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221d7263-f092-4f8c-8307-a925fe4d0ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:26.017051 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-221d7263-f092-4f8c-8307-a925fe4d0ca7 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:18:26.018279 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-221d7263-f092-4f8c-8307-a925fe4d0ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:26.026532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1208/2427] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:18:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:18:26.030906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95b1c7-c00d-41ae-8d3f-25688a572de7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:26.041005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d95b1c7-c00d-41ae-8d3f-25688a572de7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:26.045621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c022e3-006e-47e6-ad8f-36d6af34a2f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:26.046214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c022e3-006e-47e6-ad8f-36d6af34a2f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:26.046489 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c022e3-006e-47e6-ad8f-36d6af34a2f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:26.046758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c022e3-006e-47e6-ad8f-36d6af34a2f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:26.047423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c022e3-006e-47e6-ad8f-36d6af34a2f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:18:26.047725 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c022e3-006e-47e6-ad8f-36d6af34a2f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:26.049333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c022e3-006e-47e6-ad8f-36d6af34a2f4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:26.061766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1220/2428] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:18:26.066248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d7263-f092-4f8c-8307-a925fe4d0ca7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:26.076258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221d7263-f092-4f8c-8307-a925fe4d0ca7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:26.081075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841641e-6a5a-4d2b-a8cf-3eb5359df829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:26.081641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841641e-6a5a-4d2b-a8cf-3eb5359df829 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:26.081919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841641e-6a5a-4d2b-a8cf-3eb5359df829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:26.082192 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841641e-6a5a-4d2b-a8cf-3eb5359df829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:26.082831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841641e-6a5a-4d2b-a8cf-3eb5359df829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:26.084716 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841641e-6a5a-4d2b-a8cf-3eb5359df829 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:26.110948 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1209/2429] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:18:26 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:18:26.115331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c022e3-006e-47e6-ad8f-36d6af34a2f4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:26.125974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c022e3-006e-47e6-ad8f-36d6af34a2f4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:26.184064 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66278c-f17c-4dd6-91c8-8a3ad9f976af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:26.184670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66278c-f17c-4dd6-91c8-8a3ad9f976af admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:26.184949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66278c-f17c-4dd6-91c8-8a3ad9f976af admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:26.185269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66278c-f17c-4dd6-91c8-8a3ad9f976af admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:26.185909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66278c-f17c-4dd6-91c8-8a3ad9f976af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:26.187809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66278c-f17c-4dd6-91c8-8a3ad9f976af admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:26.209155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1221/2430] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:18:26 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:18:26.213545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841641e-6a5a-4d2b-a8cf-3eb5359df829 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:26.223797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841641e-6a5a-4d2b-a8cf-3eb5359df829 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:26.228383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c5395-a97c-48c2-bbe1-6b9bdaeec5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:26.229006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c5395-a97c-48c2-bbe1-6b9bdaeec5bc admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:26.229314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c5395-a97c-48c2-bbe1-6b9bdaeec5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:26.229586 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c5395-a97c-48c2-bbe1-6b9bdaeec5bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:26.230213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c5395-a97c-48c2-bbe1-6b9bdaeec5bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:26.231820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c5395-a97c-48c2-bbe1-6b9bdaeec5bc admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:26.242166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1210/2431] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:18:26.245976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66278c-f17c-4dd6-91c8-8a3ad9f976af admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:26.256036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66278c-f17c-4dd6-91c8-8a3ad9f976af admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:26.261667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1bf420-7f82-4349-9cf0-1e25e140e065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:26.262265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bf420-7f82-4349-9cf0-1e25e140e065 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:26.262542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bf420-7f82-4349-9cf0-1e25e140e065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:26.262814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1bf420-7f82-4349-9cf0-1e25e140e065 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:26.263456 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bf420-7f82-4349-9cf0-1e25e140e065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:26.265039 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1bf420-7f82-4349-9cf0-1e25e140e065 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:26.276103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1222/2432] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:18:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:18:26.280279 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c5395-a97c-48c2-bbe1-6b9bdaeec5bc admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:26.291289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c5395-a97c-48c2-bbe1-6b9bdaeec5bc admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:26.295940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ae813-3a4d-4da8-9e3a-0a5ac0470885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:26.296576 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ae813-3a4d-4da8-9e3a-0a5ac0470885 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:26.296858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ae813-3a4d-4da8-9e3a-0a5ac0470885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:26.297217 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772ae813-3a4d-4da8-9e3a-0a5ac0470885 admin admin] PATH_INFO: `/v3/projects/eaff29c3c9b84929b0451e8e82caf4f2/users/1b01f21ffa814a5da3873e5387938b2a/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:26.312261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ae813-3a4d-4da8-9e3a-0a5ac0470885 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaff29c3c9b84929b0451e8e82caf4f2, user_id=1b01f21ffa814a5da3873e5387938b2a, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:26.315035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772ae813-3a4d-4da8-9e3a-0a5ac0470885 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:26.328082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1211/2433] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:18:26 2022] PUT /identity/v3/projects/eaff29c3c9b84929b0451e8e82caf4f2/users/1b01f21ffa814a5da3873e5387938b2a/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:18:26.333370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719f3b0-f448-4867-a3eb-b0e2d4dff2f7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:26.333869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719f3b0-f448-4867-a3eb-b0e2d4dff2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:26.333869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719f3b0-f448-4867-a3eb-b0e2d4dff2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:26.352789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2719f3b0-f448-4867-a3eb-b0e2d4dff2f7 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:18:26.354022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2719f3b0-f448-4867-a3eb-b0e2d4dff2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:26.403649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:18:26.413836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1223/2434] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:18:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:18:26.423414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ae813-3a4d-4da8-9e3a-0a5ac0470885 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:26.425301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-772ae813-3a4d-4da8-9e3a-0a5ac0470885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:26.454148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:18:26.485170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772ae813-3a4d-4da8-9e3a-0a5ac0470885 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:26.488818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a3729-6286-40b4-8f3c-746b674a597d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:26.489449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a3729-6286-40b4-8f3c-746b674a597d service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:26.489727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a3729-6286-40b4-8f3c-746b674a597d service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:26.489998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a3729-6286-40b4-8f3c-746b674a597d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:26.499587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a3729-6286-40b4-8f3c-746b674a597d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:26.501013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a3729-6286-40b4-8f3c-746b674a597d service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:26.510582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1212/2435] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:18:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:18:30.421189 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8f45-bfa5-4767-b379-45392e6ab3ca None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:30.421699 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8f45-bfa5-4767-b379-45392e6ab3ca None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:30.421808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd8f45-bfa5-4767-b379-45392e6ab3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:30.450255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-abcd8f45-bfa5-4767-b379-45392e6ab3ca None None] MFA Rules not processed for user `1b01f21ffa814a5da3873e5387938b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:18:30.451400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-abcd8f45-bfa5-4767-b379-45392e6ab3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:30.524386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1224/2436] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:18:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:18:30.539823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a3729-6286-40b4-8f3c-746b674a597d service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:30.541392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0f7a3729-6286-40b4-8f3c-746b674a597d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:30.647464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a3729-6286-40b4-8f3c-746b674a597d service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:30.665384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f80b48-2fb8-432e-8821-ee4e40f669f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:30.666080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f80b48-2fb8-432e-8821-ee4e40f669f0 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:30.666359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f80b48-2fb8-432e-8821-ee4e40f669f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:30.666632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f80b48-2fb8-432e-8821-ee4e40f669f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:30.681744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f80b48-2fb8-432e-8821-ee4e40f669f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:30.683348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f80b48-2fb8-432e-8821-ee4e40f669f0 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:30.692145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1213/2437] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:18:30 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:18:32.016975 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ceb3d-82c8-4267-bb15-6450576861fc None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:32.017538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ceb3d-82c8-4267-bb15-6450576861fc None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:32.017741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ceb3d-82c8-4267-bb15-6450576861fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:32.039402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-332ceb3d-82c8-4267-bb15-6450576861fc None None] MFA Rules not processed for user `1b01f21ffa814a5da3873e5387938b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:18:32.040615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-332ceb3d-82c8-4267-bb15-6450576861fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:32.047710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1225/2438] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:18:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:18:35.923119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f80b48-2fb8-432e-8821-ee4e40f669f0 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:18:35.924732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-94f80b48-2fb8-432e-8821-ee4e40f669f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:35.996113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f80b48-2fb8-432e-8821-ee4e40f669f0 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:18:36.001688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005a402-ad96-4931-8c7e-e62c5cf2fb74 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:18:36.002287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005a402-ad96-4931-8c7e-e62c5cf2fb74 service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:36.002565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005a402-ad96-4931-8c7e-e62c5cf2fb74 service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:36.002834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005a402-ad96-4931-8c7e-e62c5cf2fb74 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:36.004547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a005a402-ad96-4931-8c7e-e62c5cf2fb74 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:36.017617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005a402-ad96-4931-8c7e-e62c5cf2fb74 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:18:36.019233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005a402-ad96-4931-8c7e-e62c5cf2fb74 service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:18:36.029849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1214/2439] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:18:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:18:37.361670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeaeb3-e152-470e-b157-ab77dcff0ab3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:18:37.362162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeaeb3-e152-470e-b157-ab77dcff0ab3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:18:37.362261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfeaeb3-e152-470e-b157-ab77dcff0ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:18:37.381221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5cfeaeb3-e152-470e-b157-ab77dcff0ab3 None None] MFA Rules not processed for user `cc89baeb82444da78312c4b0b3fb0c41`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:18:37.382424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5cfeaeb3-e152-470e-b157-ab77dcff0ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:18:37.440316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1226/2440] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:18:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:18:46.025485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=93 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:18:47.433381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=60 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:19:06.736280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cae34-fd76-4d5d-ad96-bc226e7b8374 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:19:06.737340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cae34-fd76-4d5d-ad96-bc226e7b8374 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:19:06.737652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30cae34-fd76-4d5d-ad96-bc226e7b8374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:19:06.757062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a30cae34-fd76-4d5d-ad96-bc226e7b8374 None None] MFA Rules not processed for user `62e294148e14459a96190fab931d4898`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:19:06.758448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a30cae34-fd76-4d5d-ad96-bc226e7b8374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:19:06.812754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1215/2441] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:19:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:19:08.438681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09299c26-2c58-4667-b6b4-61ea14055f6f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:19:08.439140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09299c26-2c58-4667-b6b4-61ea14055f6f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:19:08.439238 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09299c26-2c58-4667-b6b4-61ea14055f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:19:08.459212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-09299c26-2c58-4667-b6b4-61ea14055f6f None None] MFA Rules not processed for user `bcb617f1f41c467fba9717f7f825ac80`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:19:08.460471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-09299c26-2c58-4667-b6b4-61ea14055f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:19:08.513803 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1227/2442] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:19:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:19:16.806044 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=25 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:19:18.507299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=25 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:19:28.259535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dbb41-8cfc-4d8a-9958-a1e69f3ef4f7 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:19:28.260033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dbb41-8cfc-4d8a-9958-a1e69f3ef4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:19:28.260143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669dbb41-8cfc-4d8a-9958-a1e69f3ef4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:19:28.289911 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-669dbb41-8cfc-4d8a-9958-a1e69f3ef4f7 None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:19:28.291130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-669dbb41-8cfc-4d8a-9958-a1e69f3ef4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:19:28.344307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1216/2443] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 22 22:19:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:19:28.385269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09299c26-2c58-4667-b6b4-61ea14055f6f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:19:28.386672 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-09299c26-2c58-4667-b6b4-61ea14055f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:19:28.454236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09299c26-2c58-4667-b6b4-61ea14055f6f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:19:28.459768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b302f7-dea6-4cfa-b2df-de3f5b133ba7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:19:28.460367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b302f7-dea6-4cfa-b2df-de3f5b133ba7 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:19:28.460643 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b302f7-dea6-4cfa-b2df-de3f5b133ba7 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:19:28.460920 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b302f7-dea6-4cfa-b2df-de3f5b133ba7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:19:28.468826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b302f7-dea6-4cfa-b2df-de3f5b133ba7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:19:28.470428 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b302f7-dea6-4cfa-b2df-de3f5b133ba7 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:19:28.479121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1228/2444] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:19:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:19:31.872447 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dbb41-8cfc-4d8a-9958-a1e69f3ef4f7 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:19:31.874644 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-669dbb41-8cfc-4d8a-9958-a1e69f3ef4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:19:31.891975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669dbb41-8cfc-4d8a-9958-a1e69f3ef4f7 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:19:31.898065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b73d7c-2d68-4bfa-93b3-7a3e3d79761e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:19:31.898835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b73d7c-2d68-4bfa-93b3-7a3e3d79761e service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:19:31.899180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b73d7c-2d68-4bfa-93b3-7a3e3d79761e service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:19:31.899525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b73d7c-2d68-4bfa-93b3-7a3e3d79761e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:19:31.901505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-23b73d7c-2d68-4bfa-93b3-7a3e3d79761e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:19:31.915101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b73d7c-2d68-4bfa-93b3-7a3e3d79761e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:19:31.916999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b73d7c-2d68-4bfa-93b3-7a3e3d79761e service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:19:31.934385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1217/2445] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:19:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:19:35.060150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b302f7-dea6-4cfa-b2df-de3f5b133ba7 service swift] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:19:35.061717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-01b302f7-dea6-4cfa-b2df-de3f5b133ba7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:19:35.077085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b302f7-dea6-4cfa-b2df-de3f5b133ba7 service swift] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:19:35.082041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:19:35.082679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:19:35.083132 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:19:35.083406 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] PATH_INFO: `/v3/users/9e819ca5b7e446ff86dc6a47abd88105` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:19:35.084588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e819ca5b7e446ff86dc6a47abd88105)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:19:35.086574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:19:35.161216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e819ca5b7e446ff86dc6a47abd88105'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.161864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e819ca5b7e446ff86dc6a47abd88105'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.162464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e819ca5b7e446ff86dc6a47abd88105'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.174090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e819ca5b7e446ff86dc6a47abd88105'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.181723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e819ca5b7e446ff86dc6a47abd88105'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.195228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1229/2446] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:19:35 2022] DELETE /identity/v3/users/9e819ca5b7e446ff86dc6a47abd88105 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:19:35.371108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b73d7c-2d68-4bfa-93b3-7a3e3d79761e service placement] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:19:35.388870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b73d7c-2d68-4bfa-93b3-7a3e3d79761e service placement] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:19:35.448612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:19:35.449290 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:19:35.449570 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:19:35.449844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] PATH_INFO: `/v3/projects/53f244703dfd4cb29d183025b9d6789e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:19:35.450997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f244703dfd4cb29d183025b9d6789e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:19:35.452922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:19:35.498904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f244703dfd4cb29d183025b9d6789e', 'request_id': 'req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.505540 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f244703dfd4cb29d183025b9d6789e', 'request_id': 'req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.511843 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f244703dfd4cb29d183025b9d6789e', 'request_id': 'req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.521872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] The token cache is being invalidate because project 53f244703dfd4cb29d183025b9d6789e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:19:35.522157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.522775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.524228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1218/2447] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:19:35 2022] DELETE /identity/v3/projects/53f244703dfd4cb29d183025b9d6789e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:19:35.528341 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:19:35.529821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:19:35.543241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75416d6-cb9f-4369-b4f4-a761e7f3f5cf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:19:35.598797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:19:35.599455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:19:35.599733 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:19:35.600003 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] PATH_INFO: `/v3/users/e551791f925b445686da7dee4df54891` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:19:35.601199 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e551791f925b445686da7dee4df54891)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:19:35.603172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:19:35.605500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:19:35.651877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e551791f925b445686da7dee4df54891'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.652531 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e551791f925b445686da7dee4df54891'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.653176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e551791f925b445686da7dee4df54891'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.665583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e551791f925b445686da7dee4df54891'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.672362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e551791f925b445686da7dee4df54891'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.685214 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1230/2448] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:19:35 2022] DELETE /identity/v3/users/e551791f925b445686da7dee4df54891 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:19:35.856523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:19:35.858064 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:19:35.872210 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeda7e10-b9cd-48bf-95a0-85a8fb67cebd admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:19:35.882052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:19:35.934530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:19:35.935293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:19:35.935669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:19:35.936038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] PATH_INFO: `/v3/projects/aa2609ad72ba4dc2a4c3566078265457` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:19:35.937306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa2609ad72ba4dc2a4c3566078265457)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:19:35.939290 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:19:35.989469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa2609ad72ba4dc2a4c3566078265457', 'request_id': 'req-857ced28-a50a-4fa4-9dec-07c970fa2747'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:35.997156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa2609ad72ba4dc2a4c3566078265457', 'request_id': 'req-857ced28-a50a-4fa4-9dec-07c970fa2747'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:36.004004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa2609ad72ba4dc2a4c3566078265457', 'request_id': 'req-857ced28-a50a-4fa4-9dec-07c970fa2747'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:36.013991 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] The token cache is being invalidate because project aa2609ad72ba4dc2a4c3566078265457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:19:36.014276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:36.014892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:19:36.016347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1219/2449] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:19:35 2022] DELETE /identity/v3/projects/aa2609ad72ba4dc2a4c3566078265457 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:19:45.681614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:19:46.011054 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=45,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:20:13.480178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:13.481694 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:13.557929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc17c335-c930-4bfd-8245-385296fb7200 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:13.605474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c124cc28-346f-4e65-8313-1d455282500f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:13.606071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124cc28-346f-4e65-8313-1d455282500f service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:13.606345 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124cc28-346f-4e65-8313-1d455282500f service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:13.606615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c124cc28-346f-4e65-8313-1d455282500f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:13.608314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c124cc28-346f-4e65-8313-1d455282500f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:13.676543 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124cc28-346f-4e65-8313-1d455282500f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:13.677992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c124cc28-346f-4e65-8313-1d455282500f service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:13.688708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1231/2450] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:20:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:20:14.871338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:14.872851 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:14.889233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ced28-a50a-4fa4-9dec-07c970fa2747 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:14.893417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44b7aa-d241-4053-bb4f-f7c70017c3ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:14.894015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44b7aa-d241-4053-bb4f-f7c70017c3ca service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:14.894291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44b7aa-d241-4053-bb4f-f7c70017c3ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:14.894584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44b7aa-d241-4053-bb4f-f7c70017c3ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:14.896277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8d44b7aa-d241-4053-bb4f-f7c70017c3ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:14.910616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44b7aa-d241-4053-bb4f-f7c70017c3ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:14.912038 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44b7aa-d241-4053-bb4f-f7c70017c3ca service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:14.922813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1220/2451] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:20:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:20:23.684014 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=83 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:20:24.918050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:20:34.508461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774ac9a-26d1-416f-ad26-c98d799c3ef6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:34.508936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774ac9a-26d1-416f-ad26-c98d799c3ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:34.509048 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774ac9a-26d1-416f-ad26-c98d799c3ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:34.561261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-7774ac9a-26d1-416f-ad26-c98d799c3ef6 None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:20:34.562496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7774ac9a-26d1-416f-ad26-c98d799c3ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:34.629761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1232/2452] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:20:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:20:34.676896 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44b7aa-d241-4053-bb4f-f7c70017c3ca service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:34.678534 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8d44b7aa-d241-4053-bb4f-f7c70017c3ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:34.770151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44b7aa-d241-4053-bb4f-f7c70017c3ca service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:34.776350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bdfd5-ffa0-4d83-b77e-557f918223bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:34.776945 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bdfd5-ffa0-4d83-b77e-557f918223bb service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:34.777261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bdfd5-ffa0-4d83-b77e-557f918223bb service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:34.777535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2bdfd5-ffa0-4d83-b77e-557f918223bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:34.786332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bdfd5-ffa0-4d83-b77e-557f918223bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:34.788162 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2bdfd5-ffa0-4d83-b77e-557f918223bb service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:34.796385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1221/2453] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:20:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:20:44.626680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=40 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:20:44.793151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=41 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:20:45.239971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774ac9a-26d1-416f-ad26-c98d799c3ef6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:45.249040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774ac9a-26d1-416f-ad26-c98d799c3ef6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:45.252503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44240d05-3b5e-4d0f-85a2-a4082e04ce1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:45.253092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44240d05-3b5e-4d0f-85a2-a4082e04ce1d service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:45.253394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44240d05-3b5e-4d0f-85a2-a4082e04ce1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:45.253660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44240d05-3b5e-4d0f-85a2-a4082e04ce1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:45.255481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-44240d05-3b5e-4d0f-85a2-a4082e04ce1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:45.265635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44240d05-3b5e-4d0f-85a2-a4082e04ce1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:45.267047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44240d05-3b5e-4d0f-85a2-a4082e04ce1d service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:45.275560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1233/2454] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:20:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:20:48.792681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bdfd5-ffa0-4d83-b77e-557f918223bb service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:48.801905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2bdfd5-ffa0-4d83-b77e-557f918223bb service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:48.805456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54914e9d-5918-4917-9055-133785c592a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:48.806042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54914e9d-5918-4917-9055-133785c592a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:48.806391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54914e9d-5918-4917-9055-133785c592a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:48.806766 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54914e9d-5918-4917-9055-133785c592a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:48.808685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-54914e9d-5918-4917-9055-133785c592a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:48.817817 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54914e9d-5918-4917-9055-133785c592a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:48.819242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54914e9d-5918-4917-9055-133785c592a5 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:48.827608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1222/2455] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:20:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:20:48.951344 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6659ad9-d90f-47e0-a4fc-c2dfd238eb36 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:48.951913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6659ad9-d90f-47e0-a4fc-c2dfd238eb36 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:48.951913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6659ad9-d90f-47e0-a4fc-c2dfd238eb36 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:48.953317 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1234/2456] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:20:48.959658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701a9ac-c72c-4726-98b6-940fbe206fc2 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:48.959946 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701a9ac-c72c-4726-98b6-940fbe206fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:48.960222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701a9ac-c72c-4726-98b6-940fbe206fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:48.999228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8701a9ac-c72c-4726-98b6-940fbe206fc2 None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:20:49.000437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8701a9ac-c72c-4726-98b6-940fbe206fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:49.010351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1223/2457] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:20:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:20:49.021776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6659ad9-d90f-47e0-a4fc-c2dfd238eb36 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:49.023463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a6659ad9-d90f-47e0-a4fc-c2dfd238eb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:49.207985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6659ad9-d90f-47e0-a4fc-c2dfd238eb36 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:49.211389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0130e3-8f28-4b36-9ab5-22693346ebd4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '195565a7695b4d9b82e0ee76181fee2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:49.211976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0130e3-8f28-4b36-9ab5-22693346ebd4 service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:49.212762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0130e3-8f28-4b36-9ab5-22693346ebd4 service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:49.213036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0130e3-8f28-4b36-9ab5-22693346ebd4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:49.222736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0130e3-8f28-4b36-9ab5-22693346ebd4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:49.224118 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0130e3-8f28-4b36-9ab5-22693346ebd4 service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:49.235304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1235/2458] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:20:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:20:49.308821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cedbd5-3a27-4e2b-b77d-6c740944ba03 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:49.309085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cedbd5-3a27-4e2b-b77d-6c740944ba03 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:49.309423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cedbd5-3a27-4e2b-b77d-6c740944ba03 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:49.310786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1224/2459] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:20:49.315734 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915152c-153c-4907-a85a-afa51fc83a26 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:49.316015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915152c-153c-4907-a85a-afa51fc83a26 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:49.316284 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915152c-153c-4907-a85a-afa51fc83a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:49.353803 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6915152c-153c-4907-a85a-afa51fc83a26 None None] MFA Rules not processed for user `18c956fa035f40fa88fbf662541121ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:20:49.355000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6915152c-153c-4907-a85a-afa51fc83a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:49.361350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1236/2460] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:20:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:20:49.374051 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cedbd5-3a27-4e2b-b77d-6c740944ba03 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:49.375783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-51cedbd5-3a27-4e2b-b77d-6c740944ba03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:49.389137 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cedbd5-3a27-4e2b-b77d-6c740944ba03 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:49.392614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88d57c-229d-4990-a9b8-e1b162715863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:49.393262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88d57c-229d-4990-a9b8-e1b162715863 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:49.393541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88d57c-229d-4990-a9b8-e1b162715863 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:49.393811 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88d57c-229d-4990-a9b8-e1b162715863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:49.401181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88d57c-229d-4990-a9b8-e1b162715863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:49.402732 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88d57c-229d-4990-a9b8-e1b162715863 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:49.410500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1225/2461] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:20:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:20:49.499827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814c98c-9187-4953-947b-c68ff3d92b49 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:49.500112 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814c98c-9187-4953-947b-c68ff3d92b49 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:49.500409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814c98c-9187-4953-947b-c68ff3d92b49 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:49.501793 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1237/2462] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:20:49.506772 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a811d31-9766-4412-8e06-5a1153d33d7e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:49.507051 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a811d31-9766-4412-8e06-5a1153d33d7e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:49.507319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a811d31-9766-4412-8e06-5a1153d33d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:49.527570 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0a811d31-9766-4412-8e06-5a1153d33d7e None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:20:49.528751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0a811d31-9766-4412-8e06-5a1153d33d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:49.538060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1226/2463] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:20:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:20:49.548721 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814c98c-9187-4953-947b-c68ff3d92b49 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:49.560101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814c98c-9187-4953-947b-c68ff3d92b49 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:49.563756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20345f47-4fac-4ed5-8818-d13bad36247c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:49.564339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20345f47-4fac-4ed5-8818-d13bad36247c service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:49.564614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20345f47-4fac-4ed5-8818-d13bad36247c service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:49.564883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20345f47-4fac-4ed5-8818-d13bad36247c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:49.575361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20345f47-4fac-4ed5-8818-d13bad36247c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:49.576758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20345f47-4fac-4ed5-8818-d13bad36247c service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:49.587142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1238/2464] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:20:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:20:49.650245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a811d31-9766-4412-8e06-5a1153d33d7e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:49.651640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0a811d31-9766-4412-8e06-5a1153d33d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:49.722170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a811d31-9766-4412-8e06-5a1153d33d7e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:49.727236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc8abb-2ab7-43b0-a1d2-e51c9e455a87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:49.727866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc8abb-2ab7-43b0-a1d2-e51c9e455a87 service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:49.728227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc8abb-2ab7-43b0-a1d2-e51c9e455a87 service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:49.728685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc8abb-2ab7-43b0-a1d2-e51c9e455a87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:49.730510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d6dc8abb-2ab7-43b0-a1d2-e51c9e455a87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:49.741695 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc8abb-2ab7-43b0-a1d2-e51c9e455a87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:49.743259 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc8abb-2ab7-43b0-a1d2-e51c9e455a87 service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:49.753723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1227/2465] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:20:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:20:52.561173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded7fc8-5e8c-4c3f-a683-1fe8d8d63733 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:52.561173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded7fc8-5e8c-4c3f-a683-1fe8d8d63733 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:52.561943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded7fc8-5e8c-4c3f-a683-1fe8d8d63733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:52.583652 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-dded7fc8-5e8c-4c3f-a683-1fe8d8d63733 None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:20:52.585035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dded7fc8-5e8c-4c3f-a683-1fe8d8d63733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:52.594960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1239/2466] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:20:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:20:52.607612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc8abb-2ab7-43b0-a1d2-e51c9e455a87 service placement] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:52.619507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc8abb-2ab7-43b0-a1d2-e51c9e455a87 service placement] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:52.626289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78c8d6-93ce-4df1-bcb4-47439d63c7b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:52.627174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78c8d6-93ce-4df1-bcb4-47439d63c7b4 service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:52.627583 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78c8d6-93ce-4df1-bcb4-47439d63c7b4 service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:52.628020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78c8d6-93ce-4df1-bcb4-47439d63c7b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:52.648862 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78c8d6-93ce-4df1-bcb4-47439d63c7b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:52.650937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d78c8d6-93ce-4df1-bcb4-47439d63c7b4 service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:52.660222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1228/2467] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:20:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:20:52.899720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded7fc8-5e8c-4c3f-a683-1fe8d8d63733 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:52.901140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dded7fc8-5e8c-4c3f-a683-1fe8d8d63733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:52.926025 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded7fc8-5e8c-4c3f-a683-1fe8d8d63733 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:52.935015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa04544-ac61-41cd-96dc-713c0f08237c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:52.935784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04544-ac61-41cd-96dc-713c0f08237c service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:52.936125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04544-ac61-41cd-96dc-713c0f08237c service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:52.939422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04544-ac61-41cd-96dc-713c0f08237c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:52.941459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dfa04544-ac61-41cd-96dc-713c0f08237c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:53.041058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa04544-ac61-41cd-96dc-713c0f08237c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:53.043239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa04544-ac61-41cd-96dc-713c0f08237c service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:53.064302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1240/2468] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:20:52 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:20:59.688212 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78c8d6-93ce-4df1-bcb4-47439d63c7b4 service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:20:59.690409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3d78c8d6-93ce-4df1-bcb4-47439d63c7b4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:59.794265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78c8d6-93ce-4df1-bcb4-47439d63c7b4 service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:20:59.800978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60f4bf-4ece-437e-a692-b886f53ff089 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:20:59.801640 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60f4bf-4ece-437e-a692-b886f53ff089 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:20:59.801916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60f4bf-4ece-437e-a692-b886f53ff089 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:20:59.802185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60f4bf-4ece-437e-a692-b886f53ff089 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:20:59.803935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cc60f4bf-4ece-437e-a692-b886f53ff089 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:20:59.823980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:20:59.900338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60f4bf-4ece-437e-a692-b886f53ff089 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:20:59.901869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc60f4bf-4ece-437e-a692-b886f53ff089 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:20:59.913490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1229/2469] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:20:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:21:03.061264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:09.908080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=19 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:13.187733 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6e065-5957-48fa-8cd2-26a52f2006ea None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:13.188250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6e065-5957-48fa-8cd2-26a52f2006ea None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:13.188945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a6e065-5957-48fa-8cd2-26a52f2006ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:13.207524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-28a6e065-5957-48fa-8cd2-26a52f2006ea None None] MFA Rules not processed for user `cc89baeb82444da78312c4b0b3fb0c41`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:13.208711 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-28a6e065-5957-48fa-8cd2-26a52f2006ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:13.215388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1241/2470] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:21:18.063341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60f4bf-4ece-437e-a692-b886f53ff089 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:18.065467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cc60f4bf-4ece-437e-a692-b886f53ff089 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:18.099432 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60f4bf-4ece-437e-a692-b886f53ff089 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:18.114988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32412d5d-88f7-4ed8-a1f9-e607b0464f71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:18.116116 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32412d5d-88f7-4ed8-a1f9-e607b0464f71 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:18.116557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32412d5d-88f7-4ed8-a1f9-e607b0464f71 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:18.117027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32412d5d-88f7-4ed8-a1f9-e607b0464f71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:18.119831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-32412d5d-88f7-4ed8-a1f9-e607b0464f71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:18.137099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32412d5d-88f7-4ed8-a1f9-e607b0464f71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:18.139248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32412d5d-88f7-4ed8-a1f9-e607b0464f71 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:18.155577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1230/2471] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:21:20.585577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc560-6d4f-4fd4-b832-b1ca10009758 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:20.586100 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc560-6d4f-4fd4-b832-b1ca10009758 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:20.586213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abc560-6d4f-4fd4-b832-b1ca10009758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:20.605479 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-20abc560-6d4f-4fd4-b832-b1ca10009758 None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:20.606784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-20abc560-6d4f-4fd4-b832-b1ca10009758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:20.615197 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1242/2472] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 22 22:21:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:21:20.628318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32412d5d-88f7-4ed8-a1f9-e607b0464f71 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:20.639693 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32412d5d-88f7-4ed8-a1f9-e607b0464f71 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:20.644668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9549a-596a-4550-ac1d-177e1f18d08a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:20.645301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9549a-596a-4550-ac1d-177e1f18d08a service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:20.645577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9549a-596a-4550-ac1d-177e1f18d08a service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:20.645864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9549a-596a-4550-ac1d-177e1f18d08a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:20.653671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9549a-596a-4550-ac1d-177e1f18d08a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:20.655226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9549a-596a-4550-ac1d-177e1f18d08a service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:20.663269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1231/2473] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:21:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:21:30.599769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:30.660050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=12 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:33.792513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abc560-6d4f-4fd4-b832-b1ca10009758 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:33.806849 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abc560-6d4f-4fd4-b832-b1ca10009758 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:33.812745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077741c-7046-48c0-a40b-5e15ea6838b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:33.813400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077741c-7046-48c0-a40b-5e15ea6838b4 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:33.813679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077741c-7046-48c0-a40b-5e15ea6838b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:33.813948 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b077741c-7046-48c0-a40b-5e15ea6838b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:33.815698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b077741c-7046-48c0-a40b-5e15ea6838b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:33.828943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077741c-7046-48c0-a40b-5e15ea6838b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:33.830563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b077741c-7046-48c0-a40b-5e15ea6838b4 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:33.842499 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1243/2474] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:21:37.548975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9549a-596a-4550-ac1d-177e1f18d08a service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:37.561439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9549a-596a-4550-ac1d-177e1f18d08a service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:37.566978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b182a-89c2-45d4-a875-5f1248cd6c1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:37.568083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b182a-89c2-45d4-a875-5f1248cd6c1b service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:37.568903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b182a-89c2-45d4-a875-5f1248cd6c1b service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:37.568903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977b182a-89c2-45d4-a875-5f1248cd6c1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:37.571230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-977b182a-89c2-45d4-a875-5f1248cd6c1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:37.588124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b182a-89c2-45d4-a875-5f1248cd6c1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:37.589868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977b182a-89c2-45d4-a875-5f1248cd6c1b service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:37.601889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1232/2475] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:21:38.356813 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077741c-7046-48c0-a40b-5e15ea6838b4 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:38.374081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b077741c-7046-48c0-a40b-5e15ea6838b4 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:38.379026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572fb62-2431-47bb-a144-3903956f5174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:38.381578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572fb62-2431-47bb-a144-3903956f5174 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:38.382017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572fb62-2431-47bb-a144-3903956f5174 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:38.382437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9572fb62-2431-47bb-a144-3903956f5174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:38.385239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9572fb62-2431-47bb-a144-3903956f5174 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:38.476085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572fb62-2431-47bb-a144-3903956f5174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:38.477555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9572fb62-2431-47bb-a144-3903956f5174 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:38.487602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1244/2476] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:21:44.964023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b182a-89c2-45d4-a875-5f1248cd6c1b service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:44.975977 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977b182a-89c2-45d4-a875-5f1248cd6c1b service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:44.981060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073b029-7705-404f-8667-253632a3b9fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:44.981739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073b029-7705-404f-8667-253632a3b9fd service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:44.982015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073b029-7705-404f-8667-253632a3b9fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:44.982283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073b029-7705-404f-8667-253632a3b9fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:44.984202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0073b029-7705-404f-8667-253632a3b9fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:45.057984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073b029-7705-404f-8667-253632a3b9fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:45.059436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073b029-7705-404f-8667-253632a3b9fd service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:45.073973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1233/2477] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:21:48.482986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=36 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:48.538945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572fb62-2431-47bb-a144-3903956f5174 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:48.540416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9572fb62-2431-47bb-a144-3903956f5174 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:48.559243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9572fb62-2431-47bb-a144-3903956f5174 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:48.565050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd35044b-fbf9-4597-9019-c27e378c230c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:48.565747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35044b-fbf9-4597-9019-c27e378c230c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:48.566404 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35044b-fbf9-4597-9019-c27e378c230c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:48.566813 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd35044b-fbf9-4597-9019-c27e378c230c admin admin] PATH_INFO: `/v3/users/bcb617f1f41c467fba9717f7f825ac80` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:48.568007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35044b-fbf9-4597-9019-c27e378c230c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcb617f1f41c467fba9717f7f825ac80)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:48.569974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd35044b-fbf9-4597-9019-c27e378c230c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:48.617563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fd35044b-fbf9-4597-9019-c27e378c230c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcb617f1f41c467fba9717f7f825ac80'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:48.618260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fd35044b-fbf9-4597-9019-c27e378c230c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcb617f1f41c467fba9717f7f825ac80'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:48.618854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fd35044b-fbf9-4597-9019-c27e378c230c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcb617f1f41c467fba9717f7f825ac80'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:48.631592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fd35044b-fbf9-4597-9019-c27e378c230c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcb617f1f41c467fba9717f7f825ac80'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:48.638531 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-fd35044b-fbf9-4597-9019-c27e378c230c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcb617f1f41c467fba9717f7f825ac80'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:48.656011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1245/2478] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:21:48 2022] DELETE /identity/v3/users/bcb617f1f41c467fba9717f7f825ac80 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:48.828916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073b029-7705-404f-8667-253632a3b9fd service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:48.839821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073b029-7705-404f-8667-253632a3b9fd service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:48.899871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:48.900469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:48.900743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:48.901012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] PATH_INFO: `/v3/projects/226257b343d24054bc2178b47a9a5a4f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:48.902169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=226257b343d24054bc2178b47a9a5a4f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:48.904070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:48.950853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '226257b343d24054bc2178b47a9a5a4f', 'request_id': 'req-43c1d8ec-b405-4e86-86d2-ea3fc2794226'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:48.958808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '226257b343d24054bc2178b47a9a5a4f', 'request_id': 'req-43c1d8ec-b405-4e86-86d2-ea3fc2794226'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:48.966284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '226257b343d24054bc2178b47a9a5a4f', 'request_id': 'req-43c1d8ec-b405-4e86-86d2-ea3fc2794226'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:48.977339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] The token cache is being invalidate because project 226257b343d24054bc2178b47a9a5a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:21:48.977632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:48.978276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-43c1d8ec-b405-4e86-86d2-ea3fc2794226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:48.979747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1234/2479] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:21:48 2022] DELETE /identity/v3/projects/226257b343d24054bc2178b47a9a5a4f => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:48.984637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37847d-c138-4bf9-9b5c-ab6f3828871c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:48.984920 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37847d-c138-4bf9-9b5c-ab6f3828871c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:48.985221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db37847d-c138-4bf9-9b5c-ab6f3828871c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.005009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-db37847d-c138-4bf9-9b5c-ab6f3828871c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:49.006232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-db37847d-c138-4bf9-9b5c-ab6f3828871c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:49.065326 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1246/2480] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:21:49.073602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc625b4-636a-41f0-bf04-4336fd9954ee None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.073891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc625b4-636a-41f0-bf04-4336fd9954ee None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.074170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc625b4-636a-41f0-bf04-4336fd9954ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.093757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6cc625b4-636a-41f0-bf04-4336fd9954ee None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:49.094951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6cc625b4-636a-41f0-bf04-4336fd9954ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:49.102957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1235/2481] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:21:49.107057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37847d-c138-4bf9-9b5c-ab6f3828871c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.117354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db37847d-c138-4bf9-9b5c-ab6f3828871c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.121994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380873c-2e55-4c4a-90ed-e8f8c2c5a8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.122588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380873c-2e55-4c4a-90ed-e8f8c2c5a8ce admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.122865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380873c-2e55-4c4a-90ed-e8f8c2c5a8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.123136 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380873c-2e55-4c4a-90ed-e8f8c2c5a8ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.123774 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380873c-2e55-4c4a-90ed-e8f8c2c5a8ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:21:49.124075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380873c-2e55-4c4a-90ed-e8f8c2c5a8ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.125684 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380873c-2e55-4c4a-90ed-e8f8c2c5a8ce admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.137875 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1247/2482] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:21:49.141949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc625b4-636a-41f0-bf04-4336fd9954ee None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.147589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:49.151394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc625b4-636a-41f0-bf04-4336fd9954ee None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.157130 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db8a7d-61a7-4d97-862a-47454abf7871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.157891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db8a7d-61a7-4d97-862a-47454abf7871 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.158369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db8a7d-61a7-4d97-862a-47454abf7871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.158814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db8a7d-61a7-4d97-862a-47454abf7871 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.159494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db8a7d-61a7-4d97-862a-47454abf7871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.161507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db8a7d-61a7-4d97-862a-47454abf7871 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.198847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1236/2483] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:21:49 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:21:49.202663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380873c-2e55-4c4a-90ed-e8f8c2c5a8ce admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.212273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380873c-2e55-4c4a-90ed-e8f8c2c5a8ce admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.267011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349364e-c35c-4a17-a739-5773175914bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.267595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349364e-c35c-4a17-a739-5773175914bf admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.267869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349364e-c35c-4a17-a739-5773175914bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.268140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349364e-c35c-4a17-a739-5773175914bf admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.268763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349364e-c35c-4a17-a739-5773175914bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.270674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349364e-c35c-4a17-a739-5773175914bf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.278698 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:49.294624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1248/2484] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:21:49 2022] POST /identity/v3/users => generated 283 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:21:49.298672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db8a7d-61a7-4d97-862a-47454abf7871 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.309747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db8a7d-61a7-4d97-862a-47454abf7871 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.314444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e431a66-9850-4b6f-bd01-8c1bcdde6cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.315040 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e431a66-9850-4b6f-bd01-8c1bcdde6cbe admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.315313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e431a66-9850-4b6f-bd01-8c1bcdde6cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.315580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e431a66-9850-4b6f-bd01-8c1bcdde6cbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.316218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e431a66-9850-4b6f-bd01-8c1bcdde6cbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.317850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e431a66-9850-4b6f-bd01-8c1bcdde6cbe admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.328517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1237/2485] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:49.332678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349364e-c35c-4a17-a739-5773175914bf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.342789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349364e-c35c-4a17-a739-5773175914bf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.347353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3f38c-44bf-47b6-b9f8-4f373dd96c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.347941 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f38c-44bf-47b6-b9f8-4f373dd96c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.348213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f38c-44bf-47b6-b9f8-4f373dd96c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.348477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f38c-44bf-47b6-b9f8-4f373dd96c64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.349168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3f38c-44bf-47b6-b9f8-4f373dd96c64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.350818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3f38c-44bf-47b6-b9f8-4f373dd96c64 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.361069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1249/2486] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:49.365234 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e431a66-9850-4b6f-bd01-8c1bcdde6cbe admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.375675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e431a66-9850-4b6f-bd01-8c1bcdde6cbe admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.380446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb17fc-8a40-4f73-b3ba-9fb6daa98473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.381134 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb17fc-8a40-4f73-b3ba-9fb6daa98473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.381432 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb17fc-8a40-4f73-b3ba-9fb6daa98473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.382260 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb17fc-8a40-4f73-b3ba-9fb6daa98473 admin admin] PATH_INFO: `/v3/projects/87907732b01a4afda5a90a5071bf6841/users/f222cf198a2f40779bd9a17e6a1dd895/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.396471 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb17fc-8a40-4f73-b3ba-9fb6daa98473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87907732b01a4afda5a90a5071bf6841, user_id=f222cf198a2f40779bd9a17e6a1dd895, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.399131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb17fc-8a40-4f73-b3ba-9fb6daa98473 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.412219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1238/2487] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:49 2022] PUT /identity/v3/projects/87907732b01a4afda5a90a5071bf6841/users/f222cf198a2f40779bd9a17e6a1dd895/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:49.423505 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51464b-7e5c-4e6b-b6a1-dc439d909b1b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.423896 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51464b-7e5c-4e6b-b6a1-dc439d909b1b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.424172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb51464b-7e5c-4e6b-b6a1-dc439d909b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.452459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-eb51464b-7e5c-4e6b-b6a1-dc439d909b1b None None] MFA Rules not processed for user `f222cf198a2f40779bd9a17e6a1dd895`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:49.453678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-eb51464b-7e5c-4e6b-b6a1-dc439d909b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:49.516710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1250/2488] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:21:49.521229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb17fc-8a40-4f73-b3ba-9fb6daa98473 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.583111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb17fc-8a40-4f73-b3ba-9fb6daa98473 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.587988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed204fb-9360-4493-bee7-0b844a793b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.588580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed204fb-9360-4493-bee7-0b844a793b96 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.588861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed204fb-9360-4493-bee7-0b844a793b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.589171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed204fb-9360-4493-bee7-0b844a793b96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.589797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed204fb-9360-4493-bee7-0b844a793b96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.591673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed204fb-9360-4493-bee7-0b844a793b96 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.626004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1239/2489] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:21:49 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:21:49.627086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51464b-7e5c-4e6b-b6a1-dc439d909b1b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.638385 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb51464b-7e5c-4e6b-b6a1-dc439d909b1b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.701854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be221e-b7bf-4f93-9f23-fa1a058893bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.702444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be221e-b7bf-4f93-9f23-fa1a058893bf admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.702723 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be221e-b7bf-4f93-9f23-fa1a058893bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.702995 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be221e-b7bf-4f93-9f23-fa1a058893bf admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.703626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be221e-b7bf-4f93-9f23-fa1a058893bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.705629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be221e-b7bf-4f93-9f23-fa1a058893bf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.727586 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1251/2490] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:21:49 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:21:49.731890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed204fb-9360-4493-bee7-0b844a793b96 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.743343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed204fb-9360-4493-bee7-0b844a793b96 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.748216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b0e26-9077-4223-9c11-c1e6c4f869ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.748889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b0e26-9077-4223-9c11-c1e6c4f869ec admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.749205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b0e26-9077-4223-9c11-c1e6c4f869ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.749483 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b0e26-9077-4223-9c11-c1e6c4f869ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.750137 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b0e26-9077-4223-9c11-c1e6c4f869ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.751769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b0e26-9077-4223-9c11-c1e6c4f869ec admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.763889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1240/2491] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:49.767885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be221e-b7bf-4f93-9f23-fa1a058893bf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.779253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be221e-b7bf-4f93-9f23-fa1a058893bf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.785469 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2aafb5-b519-401f-abce-4293e3ccff04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.786136 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2aafb5-b519-401f-abce-4293e3ccff04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.786475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2aafb5-b519-401f-abce-4293e3ccff04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.786683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2aafb5-b519-401f-abce-4293e3ccff04 admin admin] PATH_INFO: `/v3/projects/32a59065463f43ee82f939ade7bab733/users/cabb03d67f1d45938e5fb46b69f38859/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.804129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2aafb5-b519-401f-abce-4293e3ccff04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a59065463f43ee82f939ade7bab733, user_id=cabb03d67f1d45938e5fb46b69f38859, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.807470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2aafb5-b519-401f-abce-4293e3ccff04 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.822024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1252/2492] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:49 2022] PUT /identity/v3/projects/32a59065463f43ee82f939ade7bab733/users/cabb03d67f1d45938e5fb46b69f38859/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:49.826077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b0e26-9077-4223-9c11-c1e6c4f869ec admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.836978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b0e26-9077-4223-9c11-c1e6c4f869ec admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.895636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74698f43-d8e2-4054-afdc-6e3495ea8453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.896279 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74698f43-d8e2-4054-afdc-6e3495ea8453 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.896685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74698f43-d8e2-4054-afdc-6e3495ea8453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.896969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74698f43-d8e2-4054-afdc-6e3495ea8453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.898193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74698f43-d8e2-4054-afdc-6e3495ea8453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.899797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74698f43-d8e2-4054-afdc-6e3495ea8453 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.910034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1241/2493] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:49.914895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2aafb5-b519-401f-abce-4293e3ccff04 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:49.926637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2aafb5-b519-401f-abce-4293e3ccff04 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:49.932013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322ddc1-e856-4bbb-ad9e-3c1ad15d505e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:49.932807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322ddc1-e856-4bbb-ad9e-3c1ad15d505e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.933087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322ddc1-e856-4bbb-ad9e-3c1ad15d505e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.933392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b322ddc1-e856-4bbb-ad9e-3c1ad15d505e admin admin] PATH_INFO: `/v3/projects/32a59065463f43ee82f939ade7bab733/users/cabb03d67f1d45938e5fb46b69f38859/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.935323 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322ddc1-e856-4bbb-ad9e-3c1ad15d505e admin admin] RBAC: Authorizing `identity:create_grant(project_id=32a59065463f43ee82f939ade7bab733, user_id=cabb03d67f1d45938e5fb46b69f38859, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:49.937974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b322ddc1-e856-4bbb-ad9e-3c1ad15d505e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:49.950292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1253/2494] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:49 2022] PUT /identity/v3/projects/32a59065463f43ee82f939ade7bab733/users/cabb03d67f1d45938e5fb46b69f38859/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:49.960859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7639-92a3-4cb5-965d-f256ed2506da None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:49.961239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7639-92a3-4cb5-965d-f256ed2506da None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:49.961517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b7639-92a3-4cb5-965d-f256ed2506da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:49.982336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:49.991395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f98b7639-92a3-4cb5-965d-f256ed2506da None None] MFA Rules not processed for user `cabb03d67f1d45938e5fb46b69f38859`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:49.993135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f98b7639-92a3-4cb5-965d-f256ed2506da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:50.062395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1242/2495] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:21:50.073867 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322ddc1-e856-4bbb-ad9e-3c1ad15d505e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:50.075588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b322ddc1-e856-4bbb-ad9e-3c1ad15d505e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:50.100018 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:50.139990 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b322ddc1-e856-4bbb-ad9e-3c1ad15d505e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:50.143683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08971024-0768-41d4-aaa5-54a94f32f5c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:50.144324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08971024-0768-41d4-aaa5-54a94f32f5c6 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:50.144594 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08971024-0768-41d4-aaa5-54a94f32f5c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:50.144895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08971024-0768-41d4-aaa5-54a94f32f5c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:50.154982 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08971024-0768-41d4-aaa5-54a94f32f5c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:50.156394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08971024-0768-41d4-aaa5-54a94f32f5c6 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:50.167133 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1254/2496] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:21:50.241780 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b7639-92a3-4cb5-965d-f256ed2506da None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:50.251387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b7639-92a3-4cb5-965d-f256ed2506da None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:50.254995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8831d-fdd9-4cec-b895-11889e75315c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:50.255581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8831d-fdd9-4cec-b895-11889e75315c service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:50.255854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8831d-fdd9-4cec-b895-11889e75315c service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:50.256123 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f8831d-fdd9-4cec-b895-11889e75315c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:50.313866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8831d-fdd9-4cec-b895-11889e75315c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:50.315331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f8831d-fdd9-4cec-b895-11889e75315c service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:50.324775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1243/2497] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:21:50.920251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08971024-0768-41d4-aaa5-54a94f32f5c6 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:50.921692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-08971024-0768-41d4-aaa5-54a94f32f5c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:50.977285 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08971024-0768-41d4-aaa5-54a94f32f5c6 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:50.980839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4c51f-090c-4c92-a18b-3496c6bb2de3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:50.981463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4c51f-090c-4c92-a18b-3496c6bb2de3 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:50.981736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4c51f-090c-4c92-a18b-3496c6bb2de3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:50.982007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4c51f-090c-4c92-a18b-3496c6bb2de3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:50.983664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-25a4c51f-090c-4c92-a18b-3496c6bb2de3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:51.044018 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4c51f-090c-4c92-a18b-3496c6bb2de3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:51.045455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4c51f-090c-4c92-a18b-3496c6bb2de3 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:51.055862 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1255/2498] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:21:53.094555 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8831d-fdd9-4cec-b895-11889e75315c service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:53.105723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f8831d-fdd9-4cec-b895-11889e75315c service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:53.110282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:53.110917 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:53.111191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:53.111518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] PATH_INFO: `/v3/users/f222cf198a2f40779bd9a17e6a1dd895` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:53.112632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f222cf198a2f40779bd9a17e6a1dd895)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:53.114987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:53.165481 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f222cf198a2f40779bd9a17e6a1dd895'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:53.166250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f222cf198a2f40779bd9a17e6a1dd895'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:53.173898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f222cf198a2f40779bd9a17e6a1dd895'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:53.174627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f222cf198a2f40779bd9a17e6a1dd895'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:53.187996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f222cf198a2f40779bd9a17e6a1dd895'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:53.203488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1244/2499] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:21:53 2022] DELETE /identity/v3/users/f222cf198a2f40779bd9a17e6a1dd895 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:53.207994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb897dfe-2b43-440d-98ef-f59e346e8089 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:53.208522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb897dfe-2b43-440d-98ef-f59e346e8089 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:53.208522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb897dfe-2b43-440d-98ef-f59e346e8089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:53.229540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-eb897dfe-2b43-440d-98ef-f59e346e8089 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:53.230729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-eb897dfe-2b43-440d-98ef-f59e346e8089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:53.306234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1256/2500] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:21:53.320645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:53.322308 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:53.333877 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:53.404123 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e838c0c-8262-4867-bc13-0250dbb26779 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:53.407775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb1764-6bbc-41e3-a92b-6b7744311055 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:53.408387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb1764-6bbc-41e3-a92b-6b7744311055 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:53.408666 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb1764-6bbc-41e3-a92b-6b7744311055 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:53.408938 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb1764-6bbc-41e3-a92b-6b7744311055 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:53.418928 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb1764-6bbc-41e3-a92b-6b7744311055 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:53.420498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fb1764-6bbc-41e3-a92b-6b7744311055 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:53.431064 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1245/2501] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:21:53.871123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb897dfe-2b43-440d-98ef-f59e346e8089 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:53.872872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-eb897dfe-2b43-440d-98ef-f59e346e8089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:53.890874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb897dfe-2b43-440d-98ef-f59e346e8089 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:53.896368 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:53.897202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:53.897476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:53.897778 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] PATH_INFO: `/v3/projects/87907732b01a4afda5a90a5071bf6841` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:53.898957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87907732b01a4afda5a90a5071bf6841)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:53.901004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:53.922905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:53.978034 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87907732b01a4afda5a90a5071bf6841', 'request_id': 'req-7e91429c-836a-43da-a292-40c015abe206'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:53.986216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87907732b01a4afda5a90a5071bf6841', 'request_id': 'req-7e91429c-836a-43da-a292-40c015abe206'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:53.997670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87907732b01a4afda5a90a5071bf6841', 'request_id': 'req-7e91429c-836a-43da-a292-40c015abe206'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.016564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] The token cache is being invalidate because project 87907732b01a4afda5a90a5071bf6841 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:21:54.016945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.017646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.020243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1257/2502] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:21:53 2022] DELETE /identity/v3/projects/87907732b01a4afda5a90a5071bf6841 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:54.023943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb1764-6bbc-41e3-a92b-6b7744311055 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:54.036421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fb1764-6bbc-41e3-a92b-6b7744311055 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:54.127606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:54.128420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:54.128712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:54.128992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] PATH_INFO: `/v3/users/cabb03d67f1d45938e5fb46b69f38859` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:54.130585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] RBAC: Authorizing `identity:delete_user(user_id=cabb03d67f1d45938e5fb46b69f38859)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:54.133497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:54.219962 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cabb03d67f1d45938e5fb46b69f38859'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.220420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cabb03d67f1d45938e5fb46b69f38859'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.230510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cabb03d67f1d45938e5fb46b69f38859'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.231233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cabb03d67f1d45938e5fb46b69f38859'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.246422 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cabb03d67f1d45938e5fb46b69f38859'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.264043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1246/2503] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:21:54 2022] DELETE /identity/v3/users/cabb03d67f1d45938e5fb46b69f38859 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:54.578443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:54.579879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:54.600374 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e91429c-836a-43da-a292-40c015abe206 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:54.672831 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:54.673762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:54.674041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:54.674311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] PATH_INFO: `/v3/projects/32a59065463f43ee82f939ade7bab733` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:54.675640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=32a59065463f43ee82f939ade7bab733)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:54.677570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:54.699290 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:54.700968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:54.715194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8203df89-8ccc-4a29-b1ef-d43cebd560da admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:54.726111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:54.726754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:54.727135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:54.727410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] PATH_INFO: `/v3/users/62e294148e14459a96190fab931d4898` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:54.728581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e294148e14459a96190fab931d4898)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:54.730819 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:54.740820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32a59065463f43ee82f939ade7bab733', 'request_id': 'req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.748421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32a59065463f43ee82f939ade7bab733', 'request_id': 'req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.755167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32a59065463f43ee82f939ade7bab733', 'request_id': 'req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.770669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] The token cache is being invalidate because project 32a59065463f43ee82f939ade7bab733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:21:54.770967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.771677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-962ed34d-2a01-46ec-aa0e-e6680e5c3a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.774362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1258/2504] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:21:54 2022] DELETE /identity/v3/projects/32a59065463f43ee82f939ade7bab733 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:54.779228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a724dd-f5af-4f90-94eb-b83cc39c9ef3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:54.779516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a724dd-f5af-4f90-94eb-b83cc39c9ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:54.779794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a724dd-f5af-4f90-94eb-b83cc39c9ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:54.802342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-93a724dd-f5af-4f90-94eb-b83cc39c9ef3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:54.803872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-93a724dd-f5af-4f90-94eb-b83cc39c9ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:54.827933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:54.874929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1259/2505] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:54.878029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:54.884154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb6185-1290-401b-ab61-9b3895a1c299 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:54.884560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb6185-1290-401b-ab61-9b3895a1c299 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:54.884785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedb6185-1290-401b-ab61-9b3895a1c299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:54.905295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-bedb6185-1290-401b-ab61-9b3895a1c299 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:54.906534 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bedb6185-1290-401b-ab61-9b3895a1c299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:54.906857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e294148e14459a96190fab931d4898'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.907514 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e294148e14459a96190fab931d4898'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.914992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1260/2506] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:54.915386 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e294148e14459a96190fab931d4898'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.916034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e294148e14459a96190fab931d4898'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.919104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb6185-1290-401b-ab61-9b3895a1c299 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:54.927943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedb6185-1290-401b-ab61-9b3895a1c299 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:54.933472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ff508-71ad-4907-ad53-c5eff9dcad1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:54.934067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff508-71ad-4907-ad53-c5eff9dcad1b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:54.934380 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff508-71ad-4907-ad53-c5eff9dcad1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:54.934645 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1ff508-71ad-4907-ad53-c5eff9dcad1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:54.935279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ff508-71ad-4907-ad53-c5eff9dcad1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:21:54.935574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ff508-71ad-4907-ad53-c5eff9dcad1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:54.937160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1ff508-71ad-4907-ad53-c5eff9dcad1b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:54.938851 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e294148e14459a96190fab931d4898'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:54.949379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1261/2507] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:21:54.953319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1247/2508] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:21:54 2022] DELETE /identity/v3/users/62e294148e14459a96190fab931d4898 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:54.953581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ff508-71ad-4907-ad53-c5eff9dcad1b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:54.963392 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1ff508-71ad-4907-ad53-c5eff9dcad1b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.020949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f449e45-0139-4a76-8fd4-28e85e7d85a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.021621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449e45-0139-4a76-8fd4-28e85e7d85a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.021881 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449e45-0139-4a76-8fd4-28e85e7d85a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.022155 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449e45-0139-4a76-8fd4-28e85e7d85a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.022811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f449e45-0139-4a76-8fd4-28e85e7d85a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.024755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f449e45-0139-4a76-8fd4-28e85e7d85a9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.057270 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1262/2509] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:21:54 2022] POST /identity/v3/projects => generated 379 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:21:55.060318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.071009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235af7a2-4d37-40a9-bead-6d69418f761e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.130769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc78517-cbf8-4c91-b01b-a7ea93ea59ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.131606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc78517-cbf8-4c91-b01b-a7ea93ea59ef admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.131885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc78517-cbf8-4c91-b01b-a7ea93ea59ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.132159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc78517-cbf8-4c91-b01b-a7ea93ea59ef admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.132834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc78517-cbf8-4c91-b01b-a7ea93ea59ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.134675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f449e45-0139-4a76-8fd4-28e85e7d85a9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.134899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc78517-cbf8-4c91-b01b-a7ea93ea59ef admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.136814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6f449e45-0139-4a76-8fd4-28e85e7d85a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:55.153257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f449e45-0139-4a76-8fd4-28e85e7d85a9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.157928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.158511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.158816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.159090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] PATH_INFO: `/v3/projects/650863eae15e4733b50e22c7595d7742` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.160223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=650863eae15e4733b50e22c7595d7742)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.162140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.173209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1248/2510] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:21:55 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:21:55.174081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc78517-cbf8-4c91-b01b-a7ea93ea59ef admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.185850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc78517-cbf8-4c91-b01b-a7ea93ea59ef admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.192538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4c32c-d457-4707-b9cb-f003c79d7c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.193220 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4c32c-d457-4707-b9cb-f003c79d7c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.193527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4c32c-d457-4707-b9cb-f003c79d7c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.193799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4c32c-d457-4707-b9cb-f003c79d7c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.194477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4c32c-d457-4707-b9cb-f003c79d7c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.196131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4c32c-d457-4707-b9cb-f003c79d7c7f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.206809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1249/2511] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:55.211442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4c32c-d457-4707-b9cb-f003c79d7c7f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.215221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '650863eae15e4733b50e22c7595d7742', 'request_id': 'req-566680c3-7dee-470b-bb34-912b2abd6e64'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:55.221221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4c32c-d457-4707-b9cb-f003c79d7c7f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.221589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '650863eae15e4733b50e22c7595d7742', 'request_id': 'req-566680c3-7dee-470b-bb34-912b2abd6e64'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:55.227969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '650863eae15e4733b50e22c7595d7742', 'request_id': 'req-566680c3-7dee-470b-bb34-912b2abd6e64'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:55.240398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] The token cache is being invalidate because project 650863eae15e4733b50e22c7595d7742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:21:55.240789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:55.241671 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-566680c3-7dee-470b-bb34-912b2abd6e64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:55.243473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1263/2512] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:21:55 2022] DELETE /identity/v3/projects/650863eae15e4733b50e22c7595d7742 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:55.248345 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817485e-844d-4b78-a045-5547bb803111 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.248632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817485e-844d-4b78-a045-5547bb803111 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.248908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5817485e-844d-4b78-a045-5547bb803111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.267873 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5817485e-844d-4b78-a045-5547bb803111 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:55.269071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5817485e-844d-4b78-a045-5547bb803111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:55.279148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723f19e-f9f7-4f48-a965-dac157326e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.279771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:55.280086 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723f19e-f9f7-4f48-a965-dac157326e1a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.280417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723f19e-f9f7-4f48-a965-dac157326e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.280881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5723f19e-f9f7-4f48-a965-dac157326e1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.281968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723f19e-f9f7-4f48-a965-dac157326e1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.283955 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5723f19e-f9f7-4f48-a965-dac157326e1a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.295979 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1250/2513] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:55.299973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723f19e-f9f7-4f48-a965-dac157326e1a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.301649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5723f19e-f9f7-4f48-a965-dac157326e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:55.313910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5723f19e-f9f7-4f48-a965-dac157326e1a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.319004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3531c2-df2c-40fe-8250-d363912f7b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.319668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3531c2-df2c-40fe-8250-d363912f7b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.320072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3531c2-df2c-40fe-8250-d363912f7b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.320360 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3531c2-df2c-40fe-8250-d363912f7b0c admin admin] PATH_INFO: `/v3/projects/271ffe0e3d5c4cdabde727fcd788a3e9/users/3a5e548b7d114849b60f583359499d73/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.331792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:55.334514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1264/2514] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:55.336730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3531c2-df2c-40fe-8250-d363912f7b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=271ffe0e3d5c4cdabde727fcd788a3e9, user_id=3a5e548b7d114849b60f583359499d73, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.340315 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3531c2-df2c-40fe-8250-d363912f7b0c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.343151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceb0ee-65d0-4991-83be-22f4f41ad314 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.343431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceb0ee-65d0-4991-83be-22f4f41ad314 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.343700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ceb0ee-65d0-4991-83be-22f4f41ad314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.355421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1251/2515] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:55 2022] PUT /identity/v3/projects/271ffe0e3d5c4cdabde727fcd788a3e9/users/3a5e548b7d114849b60f583359499d73/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:55.366312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-48ceb0ee-65d0-4991-83be-22f4f41ad314 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:55.366785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff38e0-275a-4443-9352-1a5e0b5c9fcc None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.367077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff38e0-275a-4443-9352-1a5e0b5c9fcc None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.367350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff38e0-275a-4443-9352-1a5e0b5c9fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.368324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-48ceb0ee-65d0-4991-83be-22f4f41ad314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:55.394691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-faff38e0-275a-4443-9352-1a5e0b5c9fcc None None] MFA Rules not processed for user `3a5e548b7d114849b60f583359499d73`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:55.396310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-faff38e0-275a-4443-9352-1a5e0b5c9fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:55.431703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1265/2516] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:55.436116 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceb0ee-65d0-4991-83be-22f4f41ad314 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.446716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ceb0ee-65d0-4991-83be-22f4f41ad314 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.451513 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b4f3-4cb0-4b4e-bc0f-45968d8c27b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.452119 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b4f3-4cb0-4b4e-bc0f-45968d8c27b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.452399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b4f3-4cb0-4b4e-bc0f-45968d8c27b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.452666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198b4f3-4cb0-4b4e-bc0f-45968d8c27b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.453351 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198b4f3-4cb0-4b4e-bc0f-45968d8c27b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:21:55.453653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198b4f3-4cb0-4b4e-bc0f-45968d8c27b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.455232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198b4f3-4cb0-4b4e-bc0f-45968d8c27b8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.462232 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1252/2517] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:55.466440 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff38e0-275a-4443-9352-1a5e0b5c9fcc None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.470978 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1266/2518] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:21:55.471805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b4f3-4cb0-4b4e-bc0f-45968d8c27b8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.481332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff38e0-275a-4443-9352-1a5e0b5c9fcc None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.481766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198b4f3-4cb0-4b4e-bc0f-45968d8c27b8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.485345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099b523-76c6-4fe6-aa70-81a1c67cc69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.485939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099b523-76c6-4fe6-aa70-81a1c67cc69e admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.486220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fd52c-6b1d-4f3a-abeb-df97ec7ad3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.486343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099b523-76c6-4fe6-aa70-81a1c67cc69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.486559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099b523-76c6-4fe6-aa70-81a1c67cc69e admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.486792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fd52c-6b1d-4f3a-abeb-df97ec7ad3cc admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.487068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fd52c-6b1d-4f3a-abeb-df97ec7ad3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.487176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099b523-76c6-4fe6-aa70-81a1c67cc69e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.487339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755fd52c-6b1d-4f3a-abeb-df97ec7ad3cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.487954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755fd52c-6b1d-4f3a-abeb-df97ec7ad3cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.489046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099b523-76c6-4fe6-aa70-81a1c67cc69e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.489895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755fd52c-6b1d-4f3a-abeb-df97ec7ad3cc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.517508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1253/2519] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:21:55 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:55.517876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1267/2520] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:21:55 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:55.521416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099b523-76c6-4fe6-aa70-81a1c67cc69e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.521558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fd52c-6b1d-4f3a-abeb-df97ec7ad3cc admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.585180 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755fd52c-6b1d-4f3a-abeb-df97ec7ad3cc admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.586971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099b523-76c6-4fe6-aa70-81a1c67cc69e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.590535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c039edd-72c8-4b1c-b83c-8f89a55c8a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.591131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c039edd-72c8-4b1c-b83c-8f89a55c8a69 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.591409 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c039edd-72c8-4b1c-b83c-8f89a55c8a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.591620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfe726-9e66-49fc-aa56-2d15ea611dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.591834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c039edd-72c8-4b1c-b83c-8f89a55c8a69 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.592205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfe726-9e66-49fc-aa56-2d15ea611dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.592474 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c039edd-72c8-4b1c-b83c-8f89a55c8a69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.592579 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfe726-9e66-49fc-aa56-2d15ea611dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.592738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfe726-9e66-49fc-aa56-2d15ea611dc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.593457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfe726-9e66-49fc-aa56-2d15ea611dc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.594471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c039edd-72c8-4b1c-b83c-8f89a55c8a69 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.595522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfe726-9e66-49fc-aa56-2d15ea611dc8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.618593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1254/2521] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:21:55 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:21:55.619606 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1268/2522] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:21:55 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:55.622521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfe726-9e66-49fc-aa56-2d15ea611dc8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.625130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c039edd-72c8-4b1c-b83c-8f89a55c8a69 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.632984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfe726-9e66-49fc-aa56-2d15ea611dc8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.635938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c039edd-72c8-4b1c-b83c-8f89a55c8a69 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.637652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ea707-7b17-46e6-b952-16874a828314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.638236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ea707-7b17-46e6-b952-16874a828314 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.638510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ea707-7b17-46e6-b952-16874a828314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.638775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ea707-7b17-46e6-b952-16874a828314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.639403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ea707-7b17-46e6-b952-16874a828314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.640909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ae477-86d7-4521-8a9d-849b399fa3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.641093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ea707-7b17-46e6-b952-16874a828314 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.641834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ae477-86d7-4521-8a9d-849b399fa3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.642213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ae477-86d7-4521-8a9d-849b399fa3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.642542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109ae477-86d7-4521-8a9d-849b399fa3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.643183 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ae477-86d7-4521-8a9d-849b399fa3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.644842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109ae477-86d7-4521-8a9d-849b399fa3b0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.651200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:55.652903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1255/2523] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:21:55.656940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ea707-7b17-46e6-b952-16874a828314 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.658281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1269/2524] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:55.660838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ae477-86d7-4521-8a9d-849b399fa3b0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.667058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ea707-7b17-46e6-b952-16874a828314 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.671213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109ae477-86d7-4521-8a9d-849b399fa3b0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.671689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0a69a-4486-4c3a-8d6b-909d2fdf9913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.672330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0a69a-4486-4c3a-8d6b-909d2fdf9913 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.672606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0a69a-4486-4c3a-8d6b-909d2fdf9913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.672970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0a69a-4486-4c3a-8d6b-909d2fdf9913 admin admin] PATH_INFO: `/v3/projects/82bd8bdd001f4bdd887d1ef8af9073c3/users/b2d9429ab25c49adbde254988b9485aa/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.675931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69eeee-bbe6-4b7a-a910-d808055b89d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.676547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69eeee-bbe6-4b7a-a910-d808055b89d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.676824 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69eeee-bbe6-4b7a-a910-d808055b89d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.677096 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69eeee-bbe6-4b7a-a910-d808055b89d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.677796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69eeee-bbe6-4b7a-a910-d808055b89d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.679419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69eeee-bbe6-4b7a-a910-d808055b89d3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.687937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0a69a-4486-4c3a-8d6b-909d2fdf9913 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82bd8bdd001f4bdd887d1ef8af9073c3, user_id=b2d9429ab25c49adbde254988b9485aa, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.690507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0a69a-4486-4c3a-8d6b-909d2fdf9913 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.690629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1270/2525] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:21:55.695364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69eeee-bbe6-4b7a-a910-d808055b89d3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.706605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1256/2526] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:55 2022] PUT /identity/v3/projects/82bd8bdd001f4bdd887d1ef8af9073c3/users/b2d9429ab25c49adbde254988b9485aa/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:55.706799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69eeee-bbe6-4b7a-a910-d808055b89d3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.710359 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0a69a-4486-4c3a-8d6b-909d2fdf9913 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.723349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:55.766663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8012d-71ba-4a64-895d-1c691e2c81fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.767321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8012d-71ba-4a64-895d-1c691e2c81fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.767599 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8012d-71ba-4a64-895d-1c691e2c81fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.767879 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8012d-71ba-4a64-895d-1c691e2c81fb admin admin] PATH_INFO: `/v3/projects/64b3980eee3d474393179fa9d957a0f8/users/7ed752735ddf473796fd42b1a5271dee/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.774805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0a69a-4486-4c3a-8d6b-909d2fdf9913 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.779790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb056c-589b-4fa8-919d-0e514de1c033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.780492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb056c-589b-4fa8-919d-0e514de1c033 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.780765 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb056c-589b-4fa8-919d-0e514de1c033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.781032 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb056c-589b-4fa8-919d-0e514de1c033 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.781731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb056c-589b-4fa8-919d-0e514de1c033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.783161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8012d-71ba-4a64-895d-1c691e2c81fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=64b3980eee3d474393179fa9d957a0f8, user_id=7ed752735ddf473796fd42b1a5271dee, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.783364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb056c-589b-4fa8-919d-0e514de1c033 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.785948 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8012d-71ba-4a64-895d-1c691e2c81fb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.794151 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1257/2527] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:55.798594 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb056c-589b-4fa8-919d-0e514de1c033 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.799822 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1271/2528] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:55 2022] PUT /identity/v3/projects/64b3980eee3d474393179fa9d957a0f8/users/7ed752735ddf473796fd42b1a5271dee/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:55.809729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb056c-589b-4fa8-919d-0e514de1c033 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.812343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0ea1-6c92-43a2-8db3-40eb7574c1a9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.812627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0ea1-6c92-43a2-8db3-40eb7574c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.812898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0ea1-6c92-43a2-8db3-40eb7574c1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.840294 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-38aa0ea1-6c92-43a2-8db3-40eb7574c1a9 None None] MFA Rules not processed for user `7ed752735ddf473796fd42b1a5271dee`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:55.841544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-38aa0ea1-6c92-43a2-8db3-40eb7574c1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:55.870434 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd83b2-a831-40ad-92fe-ec840fa41c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.871082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd83b2-a831-40ad-92fe-ec840fa41c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.871363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd83b2-a831-40ad-92fe-ec840fa41c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.871639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd83b2-a831-40ad-92fe-ec840fa41c17 admin admin] PATH_INFO: `/v3/projects/82bd8bdd001f4bdd887d1ef8af9073c3/users/b2d9429ab25c49adbde254988b9485aa/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.873785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd83b2-a831-40ad-92fe-ec840fa41c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82bd8bdd001f4bdd887d1ef8af9073c3, user_id=b2d9429ab25c49adbde254988b9485aa, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.876470 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bd83b2-a831-40ad-92fe-ec840fa41c17 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:55.890127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1258/2529] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:55 2022] PUT /identity/v3/projects/82bd8bdd001f4bdd887d1ef8af9073c3/users/b2d9429ab25c49adbde254988b9485aa/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:55.901016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3bb42-9fde-499c-bfbf-98cb317d35ed None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.901324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3bb42-9fde-499c-bfbf-98cb317d35ed None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.901599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3bb42-9fde-499c-bfbf-98cb317d35ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.910862 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1272/2530] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:55.916121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa0ea1-6c92-43a2-8db3-40eb7574c1a9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:55.927511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa0ea1-6c92-43a2-8db3-40eb7574c1a9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:55.932904 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9fb3bb42-9fde-499c-bfbf-98cb317d35ed None None] MFA Rules not processed for user `b2d9429ab25c49adbde254988b9485aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:55.934167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9fb3bb42-9fde-499c-bfbf-98cb317d35ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:55.994683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78211a-ceb3-4241-ae37-b9ec71240cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:55.995356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78211a-ceb3-4241-ae37-b9ec71240cbf admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:55.995580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78211a-ceb3-4241-ae37-b9ec71240cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:55.995913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78211a-ceb3-4241-ae37-b9ec71240cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:55.996481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78211a-ceb3-4241-ae37-b9ec71240cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:55.998419 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb78211a-ceb3-4241-ae37-b9ec71240cbf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.004009 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:56.013775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1259/2531] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:21:56.021825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3bb42-9fde-499c-bfbf-98cb317d35ed None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.023929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9fb3bb42-9fde-499c-bfbf-98cb317d35ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:56.035992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1273/2532] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:21:55 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:56.037319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78211a-ceb3-4241-ae37-b9ec71240cbf admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.057784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:56.088474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3bb42-9fde-499c-bfbf-98cb317d35ed None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:56.105453 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78211a-ceb3-4241-ae37-b9ec71240cbf admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:56.110644 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053b9a9-1e15-46b4-b176-b2ce320908d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:56.111256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053b9a9-1e15-46b4-b176-b2ce320908d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.111535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053b9a9-1e15-46b4-b176-b2ce320908d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.111805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7053b9a9-1e15-46b4-b176-b2ce320908d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.112455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053b9a9-1e15-46b4-b176-b2ce320908d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:56.114378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7053b9a9-1e15-46b4-b176-b2ce320908d7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.138605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500109e-7920-4087-94ac-d2d80046255a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:56.139207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500109e-7920-4087-94ac-d2d80046255a service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.139458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500109e-7920-4087-94ac-d2d80046255a service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.139744 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500109e-7920-4087-94ac-d2d80046255a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.142595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1274/2533] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:21:56 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:56.143460 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053b9a9-1e15-46b4-b176-b2ce320908d7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.153613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7053b9a9-1e15-46b4-b176-b2ce320908d7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:56.158416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b9405-f94f-4659-9746-51181e6d8e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:56.159006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b9405-f94f-4659-9746-51181e6d8e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.159329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b9405-f94f-4659-9746-51181e6d8e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.159604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b9405-f94f-4659-9746-51181e6d8e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.160243 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b9405-f94f-4659-9746-51181e6d8e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:56.161883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b9405-f94f-4659-9746-51181e6d8e82 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.172246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1275/2534] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:56.176300 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b9405-f94f-4659-9746-51181e6d8e82 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.186591 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b9405-f94f-4659-9746-51181e6d8e82 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:56.191188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d248abf-3fb7-4898-b9af-c1cc47f05934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:56.191810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d248abf-3fb7-4898-b9af-c1cc47f05934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.192083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d248abf-3fb7-4898-b9af-c1cc47f05934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.192347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d248abf-3fb7-4898-b9af-c1cc47f05934 admin admin] PATH_INFO: `/v3/projects/7243cae4643e4e979e197ba873e86281/users/da46c98634fc460f82e4aac249621dfd/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.200425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500109e-7920-4087-94ac-d2d80046255a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:56.201947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500109e-7920-4087-94ac-d2d80046255a service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.207427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d248abf-3fb7-4898-b9af-c1cc47f05934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7243cae4643e4e979e197ba873e86281, user_id=da46c98634fc460f82e4aac249621dfd, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:56.209963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d248abf-3fb7-4898-b9af-c1cc47f05934 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.212299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1260/2535] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:21:56.224007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1276/2536] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:56 2022] PUT /identity/v3/projects/7243cae4643e4e979e197ba873e86281/users/da46c98634fc460f82e4aac249621dfd/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:56.228655 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500109e-7920-4087-94ac-d2d80046255a service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.239209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500109e-7920-4087-94ac-d2d80046255a service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:56.301937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6db242-a15b-41e5-81fb-438889382c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:56.302536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6db242-a15b-41e5-81fb-438889382c13 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.302809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6db242-a15b-41e5-81fb-438889382c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.303074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6db242-a15b-41e5-81fb-438889382c13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.303710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6db242-a15b-41e5-81fb-438889382c13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:56.306249 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6db242-a15b-41e5-81fb-438889382c13 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.316749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1261/2537] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:56.320760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d248abf-3fb7-4898-b9af-c1cc47f05934 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.333125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d248abf-3fb7-4898-b9af-c1cc47f05934 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:56.338153 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf539fa-fe44-4742-81e2-355f0353dbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:56.338801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf539fa-fe44-4742-81e2-355f0353dbbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.339094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf539fa-fe44-4742-81e2-355f0353dbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.339333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf539fa-fe44-4742-81e2-355f0353dbbd admin admin] PATH_INFO: `/v3/projects/7243cae4643e4e979e197ba873e86281/users/da46c98634fc460f82e4aac249621dfd/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.341442 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf539fa-fe44-4742-81e2-355f0353dbbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7243cae4643e4e979e197ba873e86281, user_id=da46c98634fc460f82e4aac249621dfd, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:56.344221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf539fa-fe44-4742-81e2-355f0353dbbd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.355777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1277/2538] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:56 2022] PUT /identity/v3/projects/7243cae4643e4e979e197ba873e86281/users/da46c98634fc460f82e4aac249621dfd/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:56.366952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39afb-97a2-497c-8c63-3c397f632582 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.367747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39afb-97a2-497c-8c63-3c397f632582 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.368024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de39afb-97a2-497c-8c63-3c397f632582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.395399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5de39afb-97a2-497c-8c63-3c397f632582 None None] MFA Rules not processed for user `da46c98634fc460f82e4aac249621dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:56.396601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5de39afb-97a2-497c-8c63-3c397f632582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:56.415415 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:56.471835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1262/2539] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:21:56.484277 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf539fa-fe44-4742-81e2-355f0353dbbd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.541161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf539fa-fe44-4742-81e2-355f0353dbbd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:56.541779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de39afb-97a2-497c-8c63-3c397f632582 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.544754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46eec5-ed0d-42a9-960c-a71e41bdfd5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:56.545381 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46eec5-ed0d-42a9-960c-a71e41bdfd5c service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.545664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46eec5-ed0d-42a9-960c-a71e41bdfd5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.545930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd46eec5-ed0d-42a9-960c-a71e41bdfd5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.555918 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46eec5-ed0d-42a9-960c-a71e41bdfd5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:56.557379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd46eec5-ed0d-42a9-960c-a71e41bdfd5c service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.566870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1278/2540] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:21:56.606716 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de39afb-97a2-497c-8c63-3c397f632582 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:56.611363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b33b2e-cbf0-4cd3-9028-17067a525ab3 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2d9429ab25c49adbde254988b9485aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82bd8bdd001f4bdd887d1ef8af9073c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:56.611964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33b2e-cbf0-4cd3-9028-17067a525ab3 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.612253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33b2e-cbf0-4cd3-9028-17067a525ab3 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.612538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b33b2e-cbf0-4cd3-9028-17067a525ab3 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.613256 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b33b2e-cbf0-4cd3-9028-17067a525ab3 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:21:56.613564 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b33b2e-cbf0-4cd3-9028-17067a525ab3 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:56.615155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b33b2e-cbf0-4cd3-9028-17067a525ab3 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.628968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1263/2541] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:21:56.633159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46eec5-ed0d-42a9-960c-a71e41bdfd5c service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.644109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd46eec5-ed0d-42a9-960c-a71e41bdfd5c service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:56.648920 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50883e57-82b0-4f2c-b56d-8b60660c95e7 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2d9429ab25c49adbde254988b9485aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82bd8bdd001f4bdd887d1ef8af9073c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:56.649618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50883e57-82b0-4f2c-b56d-8b60660c95e7 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.650066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50883e57-82b0-4f2c-b56d-8b60660c95e7 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.650183 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50883e57-82b0-4f2c-b56d-8b60660c95e7 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.650803 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50883e57-82b0-4f2c-b56d-8b60660c95e7 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:56.652661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50883e57-82b0-4f2c-b56d-8b60660c95e7 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.681140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1279/2542] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:21:56 2022] POST /identity/v3/projects => generated 355 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:21:56.684879 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b33b2e-cbf0-4cd3-9028-17067a525ab3 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.696506 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b33b2e-cbf0-4cd3-9028-17067a525ab3 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:56.771846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1eb89-6d40-4611-870c-7f73354a9f72 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2d9429ab25c49adbde254988b9485aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82bd8bdd001f4bdd887d1ef8af9073c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:56.772445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1eb89-6d40-4611-870c-7f73354a9f72 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.772716 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1eb89-6d40-4611-870c-7f73354a9f72 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.772994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1eb89-6d40-4611-870c-7f73354a9f72 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.773760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1eb89-6d40-4611-870c-7f73354a9f72 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:21:56.774058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1eb89-6d40-4611-870c-7f73354a9f72 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:56.775617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f1eb89-6d40-4611-870c-7f73354a9f72 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.787106 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1264/2543] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:21:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:21:56.829775 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50883e57-82b0-4f2c-b56d-8b60660c95e7 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.841708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50883e57-82b0-4f2c-b56d-8b60660c95e7 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:56.848785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b2d9429ab25c49adbde254988b9485aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '82bd8bdd001f4bdd887d1ef8af9073c3', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:56.849465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:56.849745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:56.850013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] PATH_INFO: `/v3/projects/735ff3c71d684377832b709fc8d69060` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:56.851178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: Authorizing `identity:delete_project(project_id=735ff3c71d684377832b709fc8d69060)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:56.853400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:56.856060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:56.909654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '735ff3c71d684377832b709fc8d69060', 'request_id': 'req-17ac3fcf-3e7a-418a-9e56-964318c70e13'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:56.918669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '735ff3c71d684377832b709fc8d69060', 'request_id': 'req-17ac3fcf-3e7a-418a-9e56-964318c70e13'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:56.925632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '735ff3c71d684377832b709fc8d69060', 'request_id': 'req-17ac3fcf-3e7a-418a-9e56-964318c70e13'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:56.941454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] The token cache is being invalidate because project 735ff3c71d684377832b709fc8d69060 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:21:56.941738 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:56.942375 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:56.943877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1280/2544] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:21:56 2022] DELETE /identity/v3/projects/735ff3c71d684377832b709fc8d69060 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:56.988549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1eb89-6d40-4611-870c-7f73354a9f72 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:56.990403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e7f1eb89-6d40-4611-870c-7f73354a9f72 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:57.037641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:57.060282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f1eb89-6d40-4611-870c-7f73354a9f72 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:57.120163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78dd2c-83fc-43a6-87b6-234e4cf22b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:57.120940 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78dd2c-83fc-43a6-87b6-234e4cf22b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:57.121272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78dd2c-83fc-43a6-87b6-234e4cf22b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:57.121543 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78dd2c-83fc-43a6-87b6-234e4cf22b12 admin admin] PATH_INFO: `/v3/users/7ed752735ddf473796fd42b1a5271dee` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:57.122696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78dd2c-83fc-43a6-87b6-234e4cf22b12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed752735ddf473796fd42b1a5271dee)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:57.124588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78dd2c-83fc-43a6-87b6-234e4cf22b12 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:57.160724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:57.162217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:57.176096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0f78dd2c-83fc-43a6-87b6-234e4cf22b12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed752735ddf473796fd42b1a5271dee'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:57.176734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0f78dd2c-83fc-43a6-87b6-234e4cf22b12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed752735ddf473796fd42b1a5271dee'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:57.177067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ac3fcf-3e7a-418a-9e56-964318c70e13 tempest-VolumeQuotaClassesTest-248128460 tempest-VolumeQuotaClassesTest-248128460-project-admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:57.182251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8bb305-d10b-4919-850b-d330583508bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:57.183026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bb305-d10b-4919-850b-d330583508bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:57.183309 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bb305-d10b-4919-850b-d330583508bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:57.183584 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8bb305-d10b-4919-850b-d330583508bb admin admin] PATH_INFO: `/v3/users/3a5e548b7d114849b60f583359499d73` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:57.184450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0f78dd2c-83fc-43a6-87b6-234e4cf22b12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed752735ddf473796fd42b1a5271dee'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:57.184709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8bb305-d10b-4919-850b-d330583508bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a5e548b7d114849b60f583359499d73)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:57.185144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0f78dd2c-83fc-43a6-87b6-234e4cf22b12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed752735ddf473796fd42b1a5271dee'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:57.186660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8bb305-d10b-4919-850b-d330583508bb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:57.198705 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0f78dd2c-83fc-43a6-87b6-234e4cf22b12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed752735ddf473796fd42b1a5271dee'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:57.216146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1265/2545] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:21:56 2022] DELETE /identity/v3/users/7ed752735ddf473796fd42b1a5271dee => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:57.221326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ddf6d-2713-4a37-bb2d-e94c0d3d805c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:57.221680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ddf6d-2713-4a37-bb2d-e94c0d3d805c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:57.222024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ddf6d-2713-4a37-bb2d-e94c0d3d805c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:57.234576 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8b8bb305-d10b-4919-850b-d330583508bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a5e548b7d114849b60f583359499d73'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:57.235160 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8b8bb305-d10b-4919-850b-d330583508bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a5e548b7d114849b60f583359499d73'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:57.235819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8b8bb305-d10b-4919-850b-d330583508bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a5e548b7d114849b60f583359499d73'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:57.240673 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-4f9ddf6d-2713-4a37-bb2d-e94c0d3d805c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:57.242004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4f9ddf6d-2713-4a37-bb2d-e94c0d3d805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:57.248063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8b8bb305-d10b-4919-850b-d330583508bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a5e548b7d114849b60f583359499d73'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:57.256052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8b8bb305-d10b-4919-850b-d330583508bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a5e548b7d114849b60f583359499d73'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:57.270871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1281/2546] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:21:57 2022] DELETE /identity/v3/users/3a5e548b7d114849b60f583359499d73 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:57.275846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0877ef-f29c-452b-bb1a-4948c0a32789 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:57.276163 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0877ef-f29c-452b-bb1a-4948c0a32789 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:57.276443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0877ef-f29c-452b-bb1a-4948c0a32789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:57.295100 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8a0877ef-f29c-452b-bb1a-4948c0a32789 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:57.296295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8a0877ef-f29c-452b-bb1a-4948c0a32789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:57.306477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1266/2547] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:21:57.316448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ddf6d-2713-4a37-bb2d-e94c0d3d805c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:57.317818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-4f9ddf6d-2713-4a37-bb2d-e94c0d3d805c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:57.328848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:57.356528 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1282/2548] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:57.368785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0877ef-f29c-452b-bb1a-4948c0a32789 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:57.370386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8a0877ef-f29c-452b-bb1a-4948c0a32789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:57.376556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:57.378476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ddf6d-2713-4a37-bb2d-e94c0d3d805c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:57.426850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59075d-ea04-4a8f-87cd-7c8acae57cf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:57.427463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59075d-ea04-4a8f-87cd-7c8acae57cf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:57.427737 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59075d-ea04-4a8f-87cd-7c8acae57cf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:57.428005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a59075d-ea04-4a8f-87cd-7c8acae57cf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:57.430873 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0877ef-f29c-452b-bb1a-4948c0a32789 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:57.434651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20fcff-514c-46cf-8dea-89fa8bbb4973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:57.435245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20fcff-514c-46cf-8dea-89fa8bbb4973 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:57.435516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20fcff-514c-46cf-8dea-89fa8bbb4973 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:57.435782 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a20fcff-514c-46cf-8dea-89fa8bbb4973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:57.438114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59075d-ea04-4a8f-87cd-7c8acae57cf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:57.439533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a59075d-ea04-4a8f-87cd-7c8acae57cf1 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:57.445784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20fcff-514c-46cf-8dea-89fa8bbb4973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:57.447196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a20fcff-514c-46cf-8dea-89fa8bbb4973 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:57.449724 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1267/2549] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:21:57.457261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1283/2550] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:21:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:21:57.907457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59075d-ea04-4a8f-87cd-7c8acae57cf1 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:57.909393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8a59075d-ea04-4a8f-87cd-7c8acae57cf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:57.920660 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20fcff-514c-46cf-8dea-89fa8bbb4973 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:57.922237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5a20fcff-514c-46cf-8dea-89fa8bbb4973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:57.925236 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a59075d-ea04-4a8f-87cd-7c8acae57cf1 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:57.931891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:57.932881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:57.932881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:57.933119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] PATH_INFO: `/v3/projects/64b3980eee3d474393179fa9d957a0f8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:57.934303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64b3980eee3d474393179fa9d957a0f8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:57.936217 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:57.941443 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a20fcff-514c-46cf-8dea-89fa8bbb4973 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:57.947105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:57.948193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:57.948454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:57.949008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] PATH_INFO: `/v3/projects/271ffe0e3d5c4cdabde727fcd788a3e9` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:57.950079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=271ffe0e3d5c4cdabde727fcd788a3e9)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:57.952172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:57.993227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64b3980eee3d474393179fa9d957a0f8', 'request_id': 'req-883ff2be-193f-497b-971b-50cfbc8eff72'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.003754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64b3980eee3d474393179fa9d957a0f8', 'request_id': 'req-883ff2be-193f-497b-971b-50cfbc8eff72'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.009621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '271ffe0e3d5c4cdabde727fcd788a3e9', 'request_id': 'req-b1b3c691-ad2c-4920-9fb6-714910ecbc82'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.014829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64b3980eee3d474393179fa9d957a0f8', 'request_id': 'req-883ff2be-193f-497b-971b-50cfbc8eff72'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.016493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '271ffe0e3d5c4cdabde727fcd788a3e9', 'request_id': 'req-b1b3c691-ad2c-4920-9fb6-714910ecbc82'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.023183 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '271ffe0e3d5c4cdabde727fcd788a3e9', 'request_id': 'req-b1b3c691-ad2c-4920-9fb6-714910ecbc82'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.031090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] The token cache is being invalidate because project 64b3980eee3d474393179fa9d957a0f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:21:58.031445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.032264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.033967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1268/2551] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:21:57 2022] DELETE /identity/v3/projects/64b3980eee3d474393179fa9d957a0f8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:58.038821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:58.041420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:58.043539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] The token cache is being invalidate because project 271ffe0e3d5c4cdabde727fcd788a3e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:21:58.043836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.044858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.049670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1284/2552] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:21:57 2022] DELETE /identity/v3/projects/271ffe0e3d5c4cdabde727fcd788a3e9 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:58.056998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:58.058445 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:58.110184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:58.127627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ff2be-193f-497b-971b-50cfbc8eff72 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:58.145747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b3c691-ad2c-4920-9fb6-714910ecbc82 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:58.151168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:58.151797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.152076 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.152379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] PATH_INFO: `/v3/users/b2d9429ab25c49adbde254988b9485aa` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.153552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d9429ab25c49adbde254988b9485aa)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:58.155448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:58.188439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:58.189201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.189614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.189892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] PATH_INFO: `/v3/users/da46c98634fc460f82e4aac249621dfd` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.191243 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da46c98634fc460f82e4aac249621dfd)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:58.193450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:58.193933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:58.207949 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d9429ab25c49adbde254988b9485aa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.208624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d9429ab25c49adbde254988b9485aa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.209280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d9429ab25c49adbde254988b9485aa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.222226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d9429ab25c49adbde254988b9485aa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.230134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d9429ab25c49adbde254988b9485aa'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.242997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1285/2553] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:21:58 2022] DELETE /identity/v3/users/b2d9429ab25c49adbde254988b9485aa => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:58.243161 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da46c98634fc460f82e4aac249621dfd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.245871 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da46c98634fc460f82e4aac249621dfd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.253468 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da46c98634fc460f82e4aac249621dfd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.256327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da46c98634fc460f82e4aac249621dfd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.270847 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da46c98634fc460f82e4aac249621dfd'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.285381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1269/2554] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:21:58 2022] DELETE /identity/v3/users/da46c98634fc460f82e4aac249621dfd => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:58.581054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:58.582524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:58.629313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:58.632193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:58.653498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87629955-e8ea-4758-9ece-e51a3fc78fb3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:58.708525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13690a5a-6290-46f3-8247-39955127f456 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:58.711378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:58.712122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.712546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.712881 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] PATH_INFO: `/v3/projects/82bd8bdd001f4bdd887d1ef8af9073c3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.713063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:58.713699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.713969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.714071 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82bd8bdd001f4bdd887d1ef8af9073c3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:58.714234 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] PATH_INFO: `/v3/projects/7243cae4643e4e979e197ba873e86281` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.715324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7243cae4643e4e979e197ba873e86281)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:58.715954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:58.717255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:58.738512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:58.769360 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7243cae4643e4e979e197ba873e86281', 'request_id': 'req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.774943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82bd8bdd001f4bdd887d1ef8af9073c3', 'request_id': 'req-b1f94502-b903-4b05-9dcc-f1f3f779ce61'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.777909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7243cae4643e4e979e197ba873e86281', 'request_id': 'req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.781438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82bd8bdd001f4bdd887d1ef8af9073c3', 'request_id': 'req-b1f94502-b903-4b05-9dcc-f1f3f779ce61'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.784452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7243cae4643e4e979e197ba873e86281', 'request_id': 'req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.785284 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:58.788662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82bd8bdd001f4bdd887d1ef8af9073c3', 'request_id': 'req-b1f94502-b903-4b05-9dcc-f1f3f779ce61'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.799161 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] The token cache is being invalidate because project 7243cae4643e4e979e197ba873e86281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:21:58.799458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.800101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ea1b7cdb-48ec-482f-ab8a-149b44b7802c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.801628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1270/2555] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:21:58 2022] DELETE /identity/v3/projects/7243cae4643e4e979e197ba873e86281 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:58.803340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] The token cache is being invalidate because project 82bd8bdd001f4bdd887d1ef8af9073c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:21:58.803631 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.804280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b1f94502-b903-4b05-9dcc-f1f3f779ce61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:21:58.806842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1286/2556] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:21:58 2022] DELETE /identity/v3/projects/82bd8bdd001f4bdd887d1ef8af9073c3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:21:58.807412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f482c-6b0b-43f5-bd05-c9093bb30a90 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.807696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f482c-6b0b-43f5-bd05-c9093bb30a90 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.807966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f482c-6b0b-43f5-bd05-c9093bb30a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.811233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649b673-594a-4c91-9fb1-2d3dd81be26d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.811514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649b673-594a-4c91-9fb1-2d3dd81be26d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.811794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649b673-594a-4c91-9fb1-2d3dd81be26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.827410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-5e0f482c-6b0b-43f5-bd05-c9093bb30a90 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:58.828610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5e0f482c-6b0b-43f5-bd05-c9093bb30a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:58.831427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5649b673-594a-4c91-9fb1-2d3dd81be26d None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:58.832657 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5649b673-594a-4c91-9fb1-2d3dd81be26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:58.889018 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1271/2557] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:58.894704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1287/2558] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:58.896738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b449f8-4ca9-4e79-a1a9-75124e7af044 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.897022 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b449f8-4ca9-4e79-a1a9-75124e7af044 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.897348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b449f8-4ca9-4e79-a1a9-75124e7af044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.904270 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26e7bc-2dd7-416e-b79c-9b691d58f4f6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.904552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26e7bc-2dd7-416e-b79c-9b691d58f4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.904828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26e7bc-2dd7-416e-b79c-9b691d58f4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.918021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e8b449f8-4ca9-4e79-a1a9-75124e7af044 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:58.919268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e8b449f8-4ca9-4e79-a1a9-75124e7af044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:58.924732 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2d26e7bc-2dd7-416e-b79c-9b691d58f4f6 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:58.925953 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2d26e7bc-2dd7-416e-b79c-9b691d58f4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:58.927372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1272/2559] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:21:58.932830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b449f8-4ca9-4e79-a1a9-75124e7af044 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:58.934758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1288/2560] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:58.939379 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26e7bc-2dd7-416e-b79c-9b691d58f4f6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:58.942313 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b449f8-4ca9-4e79-a1a9-75124e7af044 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:58.948589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ce469-03e9-4760-a389-c9d70712d889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:58.949219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ce469-03e9-4760-a389-c9d70712d889 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.949389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26e7bc-2dd7-416e-b79c-9b691d58f4f6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:58.949508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ce469-03e9-4760-a389-c9d70712d889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.949752 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ce469-03e9-4760-a389-c9d70712d889 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.950421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ce469-03e9-4760-a389-c9d70712d889 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:21:58.950720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ce469-03e9-4760-a389-c9d70712d889 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:58.952340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514ce469-03e9-4760-a389-c9d70712d889 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:58.954322 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89228c-4248-4c22-8a82-3b365ad493c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:58.955477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89228c-4248-4c22-8a82-3b365ad493c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.955823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89228c-4248-4c22-8a82-3b365ad493c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.956161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f89228c-4248-4c22-8a82-3b365ad493c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.956872 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89228c-4248-4c22-8a82-3b365ad493c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:21:58.957280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89228c-4248-4c22-8a82-3b365ad493c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:58.959403 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f89228c-4248-4c22-8a82-3b365ad493c8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:58.963821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1273/2561] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:21:58.968155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ce469-03e9-4760-a389-c9d70712d889 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:58.972041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1289/2562] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:21:58.976082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89228c-4248-4c22-8a82-3b365ad493c8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:58.978896 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514ce469-03e9-4760-a389-c9d70712d889 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:58.983639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0f292-ee85-4e9c-8487-4017b7bd9776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:58.984402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0f292-ee85-4e9c-8487-4017b7bd9776 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.984679 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0f292-ee85-4e9c-8487-4017b7bd9776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.984945 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b0f292-ee85-4e9c-8487-4017b7bd9776 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.985596 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0f292-ee85-4e9c-8487-4017b7bd9776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:58.985712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f89228c-4248-4c22-8a82-3b365ad493c8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:58.987544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b0f292-ee85-4e9c-8487-4017b7bd9776 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:58.990234 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873177d-a715-440b-9a39-05e9954460bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:58.990841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873177d-a715-440b-9a39-05e9954460bc admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:58.991123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873177d-a715-440b-9a39-05e9954460bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:58.991393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873177d-a715-440b-9a39-05e9954460bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:58.992146 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873177d-a715-440b-9a39-05e9954460bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:58.994577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873177d-a715-440b-9a39-05e9954460bc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.016218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1274/2563] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:21:58 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:21:59.020546 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0f292-ee85-4e9c-8487-4017b7bd9776 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.023434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1290/2564] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:21:58 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:59.027329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873177d-a715-440b-9a39-05e9954460bc admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.084412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b0f292-ee85-4e9c-8487-4017b7bd9776 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.093345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:59.096928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873177d-a715-440b-9a39-05e9954460bc admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.102209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f730b-4d68-41da-aec2-c28518e03b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.102936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f730b-4d68-41da-aec2-c28518e03b99 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.103221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f730b-4d68-41da-aec2-c28518e03b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.103527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f730b-4d68-41da-aec2-c28518e03b99 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.104161 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f730b-4d68-41da-aec2-c28518e03b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.106108 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8f730b-4d68-41da-aec2-c28518e03b99 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.129625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1291/2565] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:21:59 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:59.134073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f730b-4d68-41da-aec2-c28518e03b99 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.146399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcce959-9848-493d-9803-49a26cbe8b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.146677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f730b-4d68-41da-aec2-c28518e03b99 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.147046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcce959-9848-493d-9803-49a26cbe8b05 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.147322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcce959-9848-493d-9803-49a26cbe8b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.147600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcce959-9848-493d-9803-49a26cbe8b05 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.148229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcce959-9848-493d-9803-49a26cbe8b05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.150435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcce959-9848-493d-9803-49a26cbe8b05 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.152260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f33030-22d0-438a-ada7-cb5995d4bda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.152863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f33030-22d0-438a-ada7-cb5995d4bda8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.153173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f33030-22d0-438a-ada7-cb5995d4bda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.153449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f33030-22d0-438a-ada7-cb5995d4bda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.154098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f33030-22d0-438a-ada7-cb5995d4bda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.155661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f33030-22d0-438a-ada7-cb5995d4bda8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.166397 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1292/2566] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:59.170266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f33030-22d0-438a-ada7-cb5995d4bda8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.174608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1275/2567] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:21:59 2022] POST /identity/v3/users => generated 278 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:59.175641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcce959-9848-493d-9803-49a26cbe8b05 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.181424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f33030-22d0-438a-ada7-cb5995d4bda8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.186216 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085bfb3-8bde-4a14-8e24-186d209596b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.186435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcce959-9848-493d-9803-49a26cbe8b05 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.186811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085bfb3-8bde-4a14-8e24-186d209596b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.187086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085bfb3-8bde-4a14-8e24-186d209596b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.187401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085bfb3-8bde-4a14-8e24-186d209596b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.187974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085bfb3-8bde-4a14-8e24-186d209596b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.189598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4085bfb3-8bde-4a14-8e24-186d209596b7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.191138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b2f45-15d0-4c97-a7d5-4419f4c7ab5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.191723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b2f45-15d0-4c97-a7d5-4419f4c7ab5b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.192004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b2f45-15d0-4c97-a7d5-4419f4c7ab5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.192278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519b2f45-15d0-4c97-a7d5-4419f4c7ab5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.192907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b2f45-15d0-4c97-a7d5-4419f4c7ab5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.194523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519b2f45-15d0-4c97-a7d5-4419f4c7ab5b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.201168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1293/2568] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:21:59.205149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085bfb3-8bde-4a14-8e24-186d209596b7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.205283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1276/2569] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:21:59.208550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b2f45-15d0-4c97-a7d5-4419f4c7ab5b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.214868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085bfb3-8bde-4a14-8e24-186d209596b7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.218072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519b2f45-15d0-4c97-a7d5-4419f4c7ab5b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.219527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71362052-f462-4577-8dae-c872f2c44f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.220212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71362052-f462-4577-8dae-c872f2c44f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.220431 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71362052-f462-4577-8dae-c872f2c44f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.220707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71362052-f462-4577-8dae-c872f2c44f33 admin admin] PATH_INFO: `/v3/projects/30192c73c0bc404788a19060fba5b18c/users/0f8f2d36307440bdab75fce0475ff8a4/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.223305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407c87c-7fda-4cda-b827-9095e75e3495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.223878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407c87c-7fda-4cda-b827-9095e75e3495 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.224153 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407c87c-7fda-4cda-b827-9095e75e3495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.224420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407c87c-7fda-4cda-b827-9095e75e3495 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.225040 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407c87c-7fda-4cda-b827-9095e75e3495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.227185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407c87c-7fda-4cda-b827-9095e75e3495 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.232877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:59.237577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1277/2570] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:21:59.237756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71362052-f462-4577-8dae-c872f2c44f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30192c73c0bc404788a19060fba5b18c, user_id=0f8f2d36307440bdab75fce0475ff8a4, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.240610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71362052-f462-4577-8dae-c872f2c44f33 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.241711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407c87c-7fda-4cda-b827-9095e75e3495 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.253224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407c87c-7fda-4cda-b827-9095e75e3495 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.256454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1294/2571] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:59 2022] PUT /identity/v3/projects/30192c73c0bc404788a19060fba5b18c/users/0f8f2d36307440bdab75fce0475ff8a4/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:59.266361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2394e81-fa57-41ca-8898-4568537066d4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.266642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2394e81-fa57-41ca-8898-4568537066d4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.266915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2394e81-fa57-41ca-8898-4568537066d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.294854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-f2394e81-fa57-41ca-8898-4568537066d4 None None] MFA Rules not processed for user `0f8f2d36307440bdab75fce0475ff8a4`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:59.296078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-f2394e81-fa57-41ca-8898-4568537066d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:59.310415 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf36c75-7540-4eb0-a4ca-d93fcf0a9a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.311001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf36c75-7540-4eb0-a4ca-d93fcf0a9a96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.311400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf36c75-7540-4eb0-a4ca-d93fcf0a9a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.311743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf36c75-7540-4eb0-a4ca-d93fcf0a9a96 admin admin] PATH_INFO: `/v3/projects/b11eb8c26f7d4b26a258c68ff91a4627/users/6ab18fbfe63d4195b45ee5fbda71ed95/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.326165 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf36c75-7540-4eb0-a4ca-d93fcf0a9a96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b11eb8c26f7d4b26a258c68ff91a4627, user_id=6ab18fbfe63d4195b45ee5fbda71ed95, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.328802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf36c75-7540-4eb0-a4ca-d93fcf0a9a96 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.342280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1278/2572] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:59 2022] PUT /identity/v3/projects/b11eb8c26f7d4b26a258c68ff91a4627/users/6ab18fbfe63d4195b45ee5fbda71ed95/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:59.354380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8cde8-9e85-4334-b9a7-a2b34ed9bf47 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.354663 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8cde8-9e85-4334-b9a7-a2b34ed9bf47 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.355019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb8cde8-9e85-4334-b9a7-a2b34ed9bf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.362040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1295/2573] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:59.366504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2394e81-fa57-41ca-8898-4568537066d4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.377351 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2394e81-fa57-41ca-8898-4568537066d4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.383147 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2cb8cde8-9e85-4334-b9a7-a2b34ed9bf47 None None] MFA Rules not processed for user `6ab18fbfe63d4195b45ee5fbda71ed95`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:59.384380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2cb8cde8-9e85-4334-b9a7-a2b34ed9bf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:59.434029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499e5fe-88a7-4854-9543-dde47320eeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.434636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499e5fe-88a7-4854-9543-dde47320eeea admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.434912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499e5fe-88a7-4854-9543-dde47320eeea admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.435191 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d499e5fe-88a7-4854-9543-dde47320eeea admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.435810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499e5fe-88a7-4854-9543-dde47320eeea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.437771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d499e5fe-88a7-4854-9543-dde47320eeea admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.438814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:59.451141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1279/2574] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:21:59.455807 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8cde8-9e85-4334-b9a7-a2b34ed9bf47 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.466067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1296/2575] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:21:59 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:59.466559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb8cde8-9e85-4334-b9a7-a2b34ed9bf47 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.469974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499e5fe-88a7-4854-9543-dde47320eeea admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.521488 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452d7fb-3d64-4700-a9dc-399335cc5ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.522290 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452d7fb-3d64-4700-a9dc-399335cc5ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.522642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452d7fb-3d64-4700-a9dc-399335cc5ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.522986 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452d7fb-3d64-4700-a9dc-399335cc5ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.523961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452d7fb-3d64-4700-a9dc-399335cc5ab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.526133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5452d7fb-3d64-4700-a9dc-399335cc5ab4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.531204 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d499e5fe-88a7-4854-9543-dde47320eeea admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.535749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0cd32-b0de-48e4-a153-a3b84d8e0f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.536373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0cd32-b0de-48e4-a153-a3b84d8e0f81 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.536649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0cd32-b0de-48e4-a153-a3b84d8e0f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.536917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0cd32-b0de-48e4-a153-a3b84d8e0f81 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.537743 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0cd32-b0de-48e4-a153-a3b84d8e0f81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.540189 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0cd32-b0de-48e4-a153-a3b84d8e0f81 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.550833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:59.557588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1280/2576] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:21:59 2022] POST /identity/v3/projects => generated 355 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:59.561898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452d7fb-3d64-4700-a9dc-399335cc5ab4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.563704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1297/2577] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:21:59 2022] POST /identity/v3/users => generated 301 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:59.568013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0cd32-b0de-48e4-a153-a3b84d8e0f81 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.578224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0cd32-b0de-48e4-a153-a3b84d8e0f81 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.624112 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452d7fb-3d64-4700-a9dc-399335cc5ab4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.628906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a434214-907d-4539-b8d5-690c817e57c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.629539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a434214-907d-4539-b8d5-690c817e57c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.629814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a434214-907d-4539-b8d5-690c817e57c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.630085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a434214-907d-4539-b8d5-690c817e57c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.630711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a434214-907d-4539-b8d5-690c817e57c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.632603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a434214-907d-4539-b8d5-690c817e57c2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.636843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf746b0-ae85-4624-a3ff-cb477c31c9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.637468 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf746b0-ae85-4624-a3ff-cb477c31c9c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.637746 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf746b0-ae85-4624-a3ff-cb477c31c9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.638041 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf746b0-ae85-4624-a3ff-cb477c31c9c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.638941 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf746b0-ae85-4624-a3ff-cb477c31c9c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.640536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf746b0-ae85-4624-a3ff-cb477c31c9c5 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.651988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1298/2578] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:21:59.655968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf746b0-ae85-4624-a3ff-cb477c31c9c5 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.658863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1281/2579] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:21:59 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:21:59.659736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a434214-907d-4539-b8d5-690c817e57c2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.666274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf746b0-ae85-4624-a3ff-cb477c31c9c5 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.670745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a434214-907d-4539-b8d5-690c817e57c2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.670947 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8f5a0-3ce4-4c43-b676-616f1e013cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.671583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8f5a0-3ce4-4c43-b676-616f1e013cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.671860 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8f5a0-3ce4-4c43-b676-616f1e013cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.672129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8f5a0-3ce4-4c43-b676-616f1e013cc1 admin admin] PATH_INFO: `/v3/projects/9113b7ff458942b4827df5637b42412d/users/bf2a193e558044afaa60dbc2d7f3f9f3/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.675461 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b9925-93c0-41d1-a878-892bec7c3d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.676043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b9925-93c0-41d1-a878-892bec7c3d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.676322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b9925-93c0-41d1-a878-892bec7c3d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.676596 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b9925-93c0-41d1-a878-892bec7c3d45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.677264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b9925-93c0-41d1-a878-892bec7c3d45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.678849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b9925-93c0-41d1-a878-892bec7c3d45 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.687602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8f5a0-3ce4-4c43-b676-616f1e013cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9113b7ff458942b4827df5637b42412d, user_id=bf2a193e558044afaa60dbc2d7f3f9f3, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.689537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1282/2580] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:21:59.690574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8f5a0-3ce4-4c43-b676-616f1e013cc1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.693537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b9925-93c0-41d1-a878-892bec7c3d45 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.703906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1299/2581] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:59 2022] PUT /identity/v3/projects/9113b7ff458942b4827df5637b42412d/users/bf2a193e558044afaa60dbc2d7f3f9f3/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:59.705094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b9925-93c0-41d1-a878-892bec7c3d45 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.707883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8f5a0-3ce4-4c43-b676-616f1e013cc1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.756777 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:59.765034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9db00e-c186-44c2-8c39-7ead31e7a9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.765793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9db00e-c186-44c2-8c39-7ead31e7a9ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.766293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9db00e-c186-44c2-8c39-7ead31e7a9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.766575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9db00e-c186-44c2-8c39-7ead31e7a9ce admin admin] PATH_INFO: `/v3/projects/096894264e3b4f8da5dadcc8e5643f2d/users/8a3cc2abdb0a4a19b5d782542c008a53/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.770845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8f5a0-3ce4-4c43-b676-616f1e013cc1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.776232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffa47d-3ad2-4ccb-a094-56cd44d2070d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.776828 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffa47d-3ad2-4ccb-a094-56cd44d2070d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.777112 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffa47d-3ad2-4ccb-a094-56cd44d2070d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.777393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffa47d-3ad2-4ccb-a094-56cd44d2070d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.778029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffa47d-3ad2-4ccb-a094-56cd44d2070d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.779631 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ffa47d-3ad2-4ccb-a094-56cd44d2070d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.781328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9db00e-c186-44c2-8c39-7ead31e7a9ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=096894264e3b4f8da5dadcc8e5643f2d, user_id=8a3cc2abdb0a4a19b5d782542c008a53, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.783945 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9db00e-c186-44c2-8c39-7ead31e7a9ce admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.793345 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1300/2582] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:21:59.796759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffa47d-3ad2-4ccb-a094-56cd44d2070d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.800033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1283/2583] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:59 2022] PUT /identity/v3/projects/096894264e3b4f8da5dadcc8e5643f2d/users/8a3cc2abdb0a4a19b5d782542c008a53/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:59.804126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9db00e-c186-44c2-8c39-7ead31e7a9ce admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.807342 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffa47d-3ad2-4ccb-a094-56cd44d2070d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.865035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbeeac-a41c-4850-91d2-3e6c64a1bb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.865299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbeeac-a41c-4850-91d2-3e6c64a1bb9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.865496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbeeac-a41c-4850-91d2-3e6c64a1bb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.865766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbeeac-a41c-4850-91d2-3e6c64a1bb9e admin admin] PATH_INFO: `/v3/projects/9113b7ff458942b4827df5637b42412d/users/bf2a193e558044afaa60dbc2d7f3f9f3/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.867833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbeeac-a41c-4850-91d2-3e6c64a1bb9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9113b7ff458942b4827df5637b42412d, user_id=bf2a193e558044afaa60dbc2d7f3f9f3, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.868675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9db00e-c186-44c2-8c39-7ead31e7a9ce admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.870522 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbeeac-a41c-4850-91d2-3e6c64a1bb9e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.873840 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58632f-228b-4b32-80b3-c6f45c854994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.874432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:21:59.874553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58632f-228b-4b32-80b3-c6f45c854994 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.874937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58632f-228b-4b32-80b3-c6f45c854994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.875283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58632f-228b-4b32-80b3-c6f45c854994 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.876077 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58632f-228b-4b32-80b3-c6f45c854994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.878392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58632f-228b-4b32-80b3-c6f45c854994 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.884362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1301/2584] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:59 2022] PUT /identity/v3/projects/9113b7ff458942b4827df5637b42412d/users/bf2a193e558044afaa60dbc2d7f3f9f3/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:59.889353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1284/2585] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:21:59.893226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbeeac-a41c-4850-91d2-3e6c64a1bb9e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:21:59.894416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b568b4-6627-4c8f-9e93-4b97b74d2900 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.894696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b568b4-6627-4c8f-9e93-4b97b74d2900 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.895071 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b568b4-6627-4c8f-9e93-4b97b74d2900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.922706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b4b568b4-6627-4c8f-9e93-4b97b74d2900 None None] MFA Rules not processed for user `bf2a193e558044afaa60dbc2d7f3f9f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:21:59.925141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b4b568b4-6627-4c8f-9e93-4b97b74d2900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:21:59.964540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbeeac-a41c-4850-91d2-3e6c64a1bb9e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:21:59.969425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c381e-ecfa-4bf9-a91f-1f3c26d7fbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:21:59.970568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c381e-ecfa-4bf9-a91f-1f3c26d7fbd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.970568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c381e-ecfa-4bf9-a91f-1f3c26d7fbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.970568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c381e-ecfa-4bf9-a91f-1f3c26d7fbd0 admin admin] PATH_INFO: `/v3/projects/096894264e3b4f8da5dadcc8e5643f2d/users/8a3cc2abdb0a4a19b5d782542c008a53/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:21:59.972487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c381e-ecfa-4bf9-a91f-1f3c26d7fbd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=096894264e3b4f8da5dadcc8e5643f2d, user_id=8a3cc2abdb0a4a19b5d782542c008a53, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:21:59.975077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c381e-ecfa-4bf9-a91f-1f3c26d7fbd0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:21:59.985909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1302/2586] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:21:59 2022] PUT /identity/v3/projects/096894264e3b4f8da5dadcc8e5643f2d/users/8a3cc2abdb0a4a19b5d782542c008a53/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:21:59.996209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeee44e-bee3-4ea2-be84-2e47703f7ee4 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:21:59.996493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeee44e-bee3-4ea2-be84-2e47703f7ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:21:59.996795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeee44e-bee3-4ea2-be84-2e47703f7ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:00.002696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1285/2587] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:59 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:00.014484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b568b4-6627-4c8f-9e93-4b97b74d2900 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:00.015845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b4b568b4-6627-4c8f-9e93-4b97b74d2900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:00.027246 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-cfeee44e-bee3-4ea2-be84-2e47703f7ee4 None None] MFA Rules not processed for user `8a3cc2abdb0a4a19b5d782542c008a53`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:00.029150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cfeee44e-bee3-4ea2-be84-2e47703f7ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:00.084500 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b568b4-6627-4c8f-9e93-4b97b74d2900 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:00.097687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:00.099846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1303/2588] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:21:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:22:00.111444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeee44e-bee3-4ea2-be84-2e47703f7ee4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:00.113351 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-cfeee44e-bee3-4ea2-be84-2e47703f7ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:00.137003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa307c2-0bd8-4a6a-b89e-7b1ac243bec7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:00.137557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa307c2-0bd8-4a6a-b89e-7b1ac243bec7 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:00.137834 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa307c2-0bd8-4a6a-b89e-7b1ac243bec7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:00.138108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa307c2-0bd8-4a6a-b89e-7b1ac243bec7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:00.172676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeee44e-bee3-4ea2-be84-2e47703f7ee4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:00.176423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a369267-fba8-4872-9e27-ab4a2bcce2fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:00.177029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a369267-fba8-4872-9e27-ab4a2bcce2fe service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:00.177344 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a369267-fba8-4872-9e27-ab4a2bcce2fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:00.177617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a369267-fba8-4872-9e27-ab4a2bcce2fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:00.187109 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a369267-fba8-4872-9e27-ab4a2bcce2fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:00.188535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a369267-fba8-4872-9e27-ab4a2bcce2fe service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:00.198581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1304/2589] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:00 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:22:00.200157 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa307c2-0bd8-4a6a-b89e-7b1ac243bec7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:00.201699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa307c2-0bd8-4a6a-b89e-7b1ac243bec7 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:00.211803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1286/2590] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:00 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:22:00.277961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a369267-fba8-4872-9e27-ab4a2bcce2fe service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:00.285898 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a369267-fba8-4872-9e27-ab4a2bcce2fe service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:00.289683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9df3a5b-c668-40ee-b6c9-e1074d71f0d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:00.290280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df3a5b-c668-40ee-b6c9-e1074d71f0d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:00.290550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df3a5b-c668-40ee-b6c9-e1074d71f0d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:00.290818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9df3a5b-c668-40ee-b6c9-e1074d71f0d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:00.304373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:00.351006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa307c2-0bd8-4a6a-b89e-7b1ac243bec7 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:00.355992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df3a5b-c668-40ee-b6c9-e1074d71f0d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:00.357530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9df3a5b-c668-40ee-b6c9-e1074d71f0d3 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:00.366462 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1305/2591] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:22:00.414497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa307c2-0bd8-4a6a-b89e-7b1ac243bec7 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:00.420169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:00.421049 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:00.421423 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:00.421689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] PATH_INFO: `/v3/users/0f8f2d36307440bdab75fce0475ff8a4` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:00.423165 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f8f2d36307440bdab75fce0475ff8a4)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:00.425055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:00.476683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f8f2d36307440bdab75fce0475ff8a4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:00.477555 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f8f2d36307440bdab75fce0475ff8a4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:00.486572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f8f2d36307440bdab75fce0475ff8a4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:00.487283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f8f2d36307440bdab75fce0475ff8a4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:00.501227 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:00.502167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f8f2d36307440bdab75fce0475ff8a4'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:00.516325 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1287/2592] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:00 2022] DELETE /identity/v3/users/0f8f2d36307440bdab75fce0475ff8a4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:00.520833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96958dc-37e1-4c30-a23c-68263e763753 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:00.521127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96958dc-37e1-4c30-a23c-68263e763753 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:00.521494 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96958dc-37e1-4c30-a23c-68263e763753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:00.540939 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d96958dc-37e1-4c30-a23c-68263e763753 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:00.542273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d96958dc-37e1-4c30-a23c-68263e763753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:00.600520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1306/2593] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:00.610477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:00.611893 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:00.668796 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6785e51-f572-490f-8c11-f326e4c878b7 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:00.672348 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f1c13-3f1b-48a9-be14-7ebb0cc26e67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:00.672936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f1c13-3f1b-48a9-be14-7ebb0cc26e67 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:00.673255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f1c13-3f1b-48a9-be14-7ebb0cc26e67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:00.673529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f1c13-3f1b-48a9-be14-7ebb0cc26e67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:00.688245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f1c13-3f1b-48a9-be14-7ebb0cc26e67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:00.689783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77f1c13-3f1b-48a9-be14-7ebb0cc26e67 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:00.700242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1288/2594] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:00.969499 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96958dc-37e1-4c30-a23c-68263e763753 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:00.971002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d96958dc-37e1-4c30-a23c-68263e763753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:00.986304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96958dc-37e1-4c30-a23c-68263e763753 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:00.991670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:00.992366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:00.993178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:00.993466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] PATH_INFO: `/v3/projects/30192c73c0bc404788a19060fba5b18c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:00.994647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30192c73c0bc404788a19060fba5b18c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:00.996486 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:01.045496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30192c73c0bc404788a19060fba5b18c', 'request_id': 'req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.051614 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30192c73c0bc404788a19060fba5b18c', 'request_id': 'req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.058627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30192c73c0bc404788a19060fba5b18c', 'request_id': 'req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.072271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] The token cache is being invalidate because project 30192c73c0bc404788a19060fba5b18c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:01.072564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.073256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.074748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1307/2595] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:00 2022] DELETE /identity/v3/projects/30192c73c0bc404788a19060fba5b18c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:01.078708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f1c13-3f1b-48a9-be14-7ebb0cc26e67 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:01.088743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f1c13-3f1b-48a9-be14-7ebb0cc26e67 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:01.143081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cf2e1-c695-4ade-bedd-9ccb736a346f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:01.143695 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cf2e1-c695-4ade-bedd-9ccb736a346f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:01.143974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cf2e1-c695-4ade-bedd-9ccb736a346f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:01.144244 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cf2e1-c695-4ade-bedd-9ccb736a346f admin admin] PATH_INFO: `/v3/users/bf2a193e558044afaa60dbc2d7f3f9f3` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:01.145381 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cf2e1-c695-4ade-bedd-9ccb736a346f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf2a193e558044afaa60dbc2d7f3f9f3)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:01.147266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39cf2e1-c695-4ade-bedd-9ccb736a346f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:01.193003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c39cf2e1-c695-4ade-bedd-9ccb736a346f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf2a193e558044afaa60dbc2d7f3f9f3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.193671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c39cf2e1-c695-4ade-bedd-9ccb736a346f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf2a193e558044afaa60dbc2d7f3f9f3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.200422 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c39cf2e1-c695-4ade-bedd-9ccb736a346f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf2a193e558044afaa60dbc2d7f3f9f3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.201074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c39cf2e1-c695-4ade-bedd-9ccb736a346f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf2a193e558044afaa60dbc2d7f3f9f3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.213401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c39cf2e1-c695-4ade-bedd-9ccb736a346f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf2a193e558044afaa60dbc2d7f3f9f3'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.226158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1289/2596] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:01 2022] DELETE /identity/v3/users/bf2a193e558044afaa60dbc2d7f3f9f3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:01.493518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:01.494955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:01.509313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce8fba9-a18f-4ecc-be7e-fa2b15b63f19 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:01.550793 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:01.572592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:01.573270 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:01.573552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:01.573825 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] PATH_INFO: `/v3/projects/9113b7ff458942b4827df5637b42412d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:01.574976 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9113b7ff458942b4827df5637b42412d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:01.576843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:01.624816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9113b7ff458942b4827df5637b42412d', 'request_id': 'req-38782085-b0cc-4cdf-a467-b64ec8ab43f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.630902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9113b7ff458942b4827df5637b42412d', 'request_id': 'req-38782085-b0cc-4cdf-a467-b64ec8ab43f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.643916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9113b7ff458942b4827df5637b42412d', 'request_id': 'req-38782085-b0cc-4cdf-a467-b64ec8ab43f2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.661012 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] The token cache is being invalidate because project 9113b7ff458942b4827df5637b42412d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:01.661389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.662209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-38782085-b0cc-4cdf-a467-b64ec8ab43f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:01.663871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1308/2597] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:01 2022] DELETE /identity/v3/projects/9113b7ff458942b4827df5637b42412d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:01.669883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee2492-1b92-4036-9e38-8e826839c0cf None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:01.670164 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee2492-1b92-4036-9e38-8e826839c0cf None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:01.670438 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee2492-1b92-4036-9e38-8e826839c0cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:01.692881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-75ee2492-1b92-4036-9e38-8e826839c0cf None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:01.694111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-75ee2492-1b92-4036-9e38-8e826839c0cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:01.698604 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:01.757050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1290/2598] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:01.765817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616edf7-90f3-49d6-b0f0-991ad6392317 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:01.766095 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616edf7-90f3-49d6-b0f0-991ad6392317 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:01.766394 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d616edf7-90f3-49d6-b0f0-991ad6392317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:01.785862 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d616edf7-90f3-49d6-b0f0-991ad6392317 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:01.787074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d616edf7-90f3-49d6-b0f0-991ad6392317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:01.795451 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1309/2599] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:01.799749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee2492-1b92-4036-9e38-8e826839c0cf None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:01.810230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee2492-1b92-4036-9e38-8e826839c0cf None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:01.814773 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d179fe1-7dd1-4749-87e6-cebdf6c848a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:01.815358 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d179fe1-7dd1-4749-87e6-cebdf6c848a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:01.815634 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d179fe1-7dd1-4749-87e6-cebdf6c848a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:01.815903 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d179fe1-7dd1-4749-87e6-cebdf6c848a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:01.816545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d179fe1-7dd1-4749-87e6-cebdf6c848a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:22:01.816837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d179fe1-7dd1-4749-87e6-cebdf6c848a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:01.818470 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d179fe1-7dd1-4749-87e6-cebdf6c848a4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:01.829688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1291/2600] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:22:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:22:01.833885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616edf7-90f3-49d6-b0f0-991ad6392317 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:01.845008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d616edf7-90f3-49d6-b0f0-991ad6392317 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:01.849628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363941b7-e24a-400b-89f1-e7b406e12285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:01.850413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363941b7-e24a-400b-89f1-e7b406e12285 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:01.850690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363941b7-e24a-400b-89f1-e7b406e12285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:01.850980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363941b7-e24a-400b-89f1-e7b406e12285 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:01.851598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363941b7-e24a-400b-89f1-e7b406e12285 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:01.854176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363941b7-e24a-400b-89f1-e7b406e12285 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:01.881297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1310/2601] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:22:01 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:01.885601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d179fe1-7dd1-4749-87e6-cebdf6c848a4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:01.895416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d179fe1-7dd1-4749-87e6-cebdf6c848a4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:01.951391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576db14-85f0-4164-8802-94e270bc4464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:01.952012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576db14-85f0-4164-8802-94e270bc4464 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:01.952278 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576db14-85f0-4164-8802-94e270bc4464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:01.952553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7576db14-85f0-4164-8802-94e270bc4464 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:01.953213 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576db14-85f0-4164-8802-94e270bc4464 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:01.955108 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7576db14-85f0-4164-8802-94e270bc4464 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:01.975913 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1292/2602] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:22:01 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:01.980750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363941b7-e24a-400b-89f1-e7b406e12285 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:01.991217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363941b7-e24a-400b-89f1-e7b406e12285 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:01.996386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e4ff6-1623-4182-9f6f-28364c4b24cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:01.996981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e4ff6-1623-4182-9f6f-28364c4b24cb admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:01.997297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e4ff6-1623-4182-9f6f-28364c4b24cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:01.997563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e4ff6-1623-4182-9f6f-28364c4b24cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:01.998200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e4ff6-1623-4182-9f6f-28364c4b24cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:02.001006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e4ff6-1623-4182-9f6f-28364c4b24cb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:02.011826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1311/2603] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:02.016982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576db14-85f0-4164-8802-94e270bc4464 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:02.028057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7576db14-85f0-4164-8802-94e270bc4464 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:02.032730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e14fa-c612-45bf-8af6-d144ac59acfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:02.033356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e14fa-c612-45bf-8af6-d144ac59acfd admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:02.033694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e14fa-c612-45bf-8af6-d144ac59acfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:02.033971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e14fa-c612-45bf-8af6-d144ac59acfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:02.034613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e14fa-c612-45bf-8af6-d144ac59acfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:02.036224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e14fa-c612-45bf-8af6-d144ac59acfd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:02.048088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1293/2604] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:02.052026 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e4ff6-1623-4182-9f6f-28364c4b24cb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:02.062923 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e4ff6-1623-4182-9f6f-28364c4b24cb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:02.068028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81198a3c-e205-4428-8fa7-5657ac5a85af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:02.068703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81198a3c-e205-4428-8fa7-5657ac5a85af admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:02.068958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81198a3c-e205-4428-8fa7-5657ac5a85af admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:02.069271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81198a3c-e205-4428-8fa7-5657ac5a85af admin admin] PATH_INFO: `/v3/projects/e52d267b52bd4f8ca0e7ea9ab2b4c9d5/users/dc981727c70346c983f532920541cc9b/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:02.084205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81198a3c-e205-4428-8fa7-5657ac5a85af admin admin] RBAC: Authorizing `identity:create_grant(project_id=e52d267b52bd4f8ca0e7ea9ab2b4c9d5, user_id=dc981727c70346c983f532920541cc9b, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:02.087753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81198a3c-e205-4428-8fa7-5657ac5a85af admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:02.101338 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1312/2605] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:02 2022] PUT /identity/v3/projects/e52d267b52bd4f8ca0e7ea9ab2b4c9d5/users/dc981727c70346c983f532920541cc9b/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:02.112291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f60007-2eb2-4810-ac53-901c84a6a5e9 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:02.112581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f60007-2eb2-4810-ac53-901c84a6a5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:02.112854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f60007-2eb2-4810-ac53-901c84a6a5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:02.140677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-33f60007-2eb2-4810-ac53-901c84a6a5e9 None None] MFA Rules not processed for user `dc981727c70346c983f532920541cc9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:02.141914 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-33f60007-2eb2-4810-ac53-901c84a6a5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:02.205391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1294/2606] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:02.217369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81198a3c-e205-4428-8fa7-5657ac5a85af admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:02.219024 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-81198a3c-e205-4428-8fa7-5657ac5a85af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:02.292463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:02.298755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81198a3c-e205-4428-8fa7-5657ac5a85af admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:02.303000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2c1a4-ce7d-4b58-aa48-04f58377f86c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '195565a7695b4d9b82e0ee76181fee2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:02.303621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c1a4-ce7d-4b58-aa48-04f58377f86c service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:02.304063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c1a4-ce7d-4b58-aa48-04f58377f86c service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:02.304339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c1a4-ce7d-4b58-aa48-04f58377f86c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:02.313077 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2c1a4-ce7d-4b58-aa48-04f58377f86c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:02.314541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2c1a4-ce7d-4b58-aa48-04f58377f86c service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:02.323933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1313/2607] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:12.202884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:12.319279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:15.331357 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f60007-2eb2-4810-ac53-901c84a6a5e9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:15.332752 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-33f60007-2eb2-4810-ac53-901c84a6a5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:15.397896 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f60007-2eb2-4810-ac53-901c84a6a5e9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:15.402768 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:15.403393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:15.403669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:15.403937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] PATH_INFO: `/v3/users/6ab18fbfe63d4195b45ee5fbda71ed95` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:15.405052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ab18fbfe63d4195b45ee5fbda71ed95)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:15.407264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:15.460872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ab18fbfe63d4195b45ee5fbda71ed95'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:15.461557 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ab18fbfe63d4195b45ee5fbda71ed95'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:15.468753 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ab18fbfe63d4195b45ee5fbda71ed95'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:15.469565 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ab18fbfe63d4195b45ee5fbda71ed95'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:15.481714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ab18fbfe63d4195b45ee5fbda71ed95'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:15.493638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1295/2608] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:15 2022] DELETE /identity/v3/users/6ab18fbfe63d4195b45ee5fbda71ed95 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:15.498622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f47f72-6017-48f9-bb93-9e5fb243d28e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:15.499130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f47f72-6017-48f9-bb93-9e5fb243d28e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:15.499250 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f47f72-6017-48f9-bb93-9e5fb243d28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:15.518434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-38f47f72-6017-48f9-bb93-9e5fb243d28e None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:15.519637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-38f47f72-6017-48f9-bb93-9e5fb243d28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:15.577043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1314/2609] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:15.586681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:15.588067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:15.644848 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2368b7-0c6c-4e01-a0f1-9c92a6faa0f5 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:15.648497 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f87b00-7048-4991-bbe9-1f5e55c55886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:15.649087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f87b00-7048-4991-bbe9-1f5e55c55886 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:15.649407 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f87b00-7048-4991-bbe9-1f5e55c55886 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:15.649682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f87b00-7048-4991-bbe9-1f5e55c55886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:15.658821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f87b00-7048-4991-bbe9-1f5e55c55886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:15.660258 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f87b00-7048-4991-bbe9-1f5e55c55886 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:15.669909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1296/2610] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:15.935465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f47f72-6017-48f9-bb93-9e5fb243d28e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:15.936893 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-38f47f72-6017-48f9-bb93-9e5fb243d28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:15.950437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f47f72-6017-48f9-bb93-9e5fb243d28e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:15.955091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:15.955720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:15.956564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:15.956853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] PATH_INFO: `/v3/projects/b11eb8c26f7d4b26a258c68ff91a4627` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:15.958180 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b11eb8c26f7d4b26a258c68ff91a4627)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:15.960065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:16.007876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b11eb8c26f7d4b26a258c68ff91a4627', 'request_id': 'req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.014321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b11eb8c26f7d4b26a258c68ff91a4627', 'request_id': 'req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.020081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b11eb8c26f7d4b26a258c68ff91a4627', 'request_id': 'req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.032535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] The token cache is being invalidate because project b11eb8c26f7d4b26a258c68ff91a4627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:16.032827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.033477 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.034980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1315/2611] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:15 2022] DELETE /identity/v3/projects/b11eb8c26f7d4b26a258c68ff91a4627 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:16.038942 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f87b00-7048-4991-bbe9-1f5e55c55886 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:16.049289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f87b00-7048-4991-bbe9-1f5e55c55886 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:16.070084 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:16.110828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17e009-dee1-4791-b796-edefd796baa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:16.111372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17e009-dee1-4791-b796-edefd796baa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:16.111645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17e009-dee1-4791-b796-edefd796baa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:16.111915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17e009-dee1-4791-b796-edefd796baa4 admin admin] PATH_INFO: `/v3/users/8a3cc2abdb0a4a19b5d782542c008a53` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:16.113057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17e009-dee1-4791-b796-edefd796baa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a3cc2abdb0a4a19b5d782542c008a53)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:16.115004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17e009-dee1-4791-b796-edefd796baa4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:16.159658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ae17e009-dee1-4791-b796-edefd796baa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a3cc2abdb0a4a19b5d782542c008a53'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.160296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ae17e009-dee1-4791-b796-edefd796baa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a3cc2abdb0a4a19b5d782542c008a53'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.167763 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ae17e009-dee1-4791-b796-edefd796baa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a3cc2abdb0a4a19b5d782542c008a53'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.168395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ae17e009-dee1-4791-b796-edefd796baa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a3cc2abdb0a4a19b5d782542c008a53'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.180265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ae17e009-dee1-4791-b796-edefd796baa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a3cc2abdb0a4a19b5d782542c008a53'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.193746 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1297/2612] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:16 2022] DELETE /identity/v3/users/8a3cc2abdb0a4a19b5d782542c008a53 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:16.455498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:16.457148 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:16.484347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d344d9d-0e78-4aa5-a8b0-6df1864f658b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:16.541626 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:16.542461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:16.542702 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:16.542965 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] PATH_INFO: `/v3/projects/096894264e3b4f8da5dadcc8e5643f2d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:16.544101 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=096894264e3b4f8da5dadcc8e5643f2d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:16.546017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:16.576187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:16.593873 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '096894264e3b4f8da5dadcc8e5643f2d', 'request_id': 'req-a532934d-5789-4469-a6fe-3c41bbfe7cb2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.599861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '096894264e3b4f8da5dadcc8e5643f2d', 'request_id': 'req-a532934d-5789-4469-a6fe-3c41bbfe7cb2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.605388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '096894264e3b4f8da5dadcc8e5643f2d', 'request_id': 'req-a532934d-5789-4469-a6fe-3c41bbfe7cb2'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.617248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] The token cache is being invalidate because project 096894264e3b4f8da5dadcc8e5643f2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:16.617538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.618153 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-a532934d-5789-4469-a6fe-3c41bbfe7cb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:16.619603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1316/2613] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:16 2022] DELETE /identity/v3/projects/096894264e3b4f8da5dadcc8e5643f2d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:16.624469 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad9c5f-fa40-4196-9419-f4b5ade82c17 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:16.624968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad9c5f-fa40-4196-9419-f4b5ade82c17 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:16.625128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad9c5f-fa40-4196-9419-f4b5ade82c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:16.644810 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-12ad9c5f-fa40-4196-9419-f4b5ade82c17 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:16.645968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-12ad9c5f-fa40-4196-9419-f4b5ade82c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:16.704574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1298/2614] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:16.713519 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476e16f-e5a5-47eb-a967-4b3f3ae6217d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:16.714520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476e16f-e5a5-47eb-a967-4b3f3ae6217d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:16.714862 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476e16f-e5a5-47eb-a967-4b3f3ae6217d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:16.735205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1476e16f-e5a5-47eb-a967-4b3f3ae6217d None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:16.736412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1476e16f-e5a5-47eb-a967-4b3f3ae6217d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:16.744279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1317/2615] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:16.749216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad9c5f-fa40-4196-9419-f4b5ade82c17 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:16.758998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad9c5f-fa40-4196-9419-f4b5ade82c17 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:16.763692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98481f63-dd03-4abe-a7e6-15288f634c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:16.764290 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98481f63-dd03-4abe-a7e6-15288f634c5a admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:16.764567 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98481f63-dd03-4abe-a7e6-15288f634c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:16.764844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98481f63-dd03-4abe-a7e6-15288f634c5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:16.765513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98481f63-dd03-4abe-a7e6-15288f634c5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:22:16.765815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98481f63-dd03-4abe-a7e6-15288f634c5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:16.767394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98481f63-dd03-4abe-a7e6-15288f634c5a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:16.779657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1299/2616] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:22:16.788150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476e16f-e5a5-47eb-a967-4b3f3ae6217d None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:16.798393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476e16f-e5a5-47eb-a967-4b3f3ae6217d None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:16.803081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1f1fd-67c2-4212-8205-13aa3c852c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:16.803674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f1fd-67c2-4212-8205-13aa3c852c33 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:16.803952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f1fd-67c2-4212-8205-13aa3c852c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:16.804219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1f1fd-67c2-4212-8205-13aa3c852c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:16.804880 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1f1fd-67c2-4212-8205-13aa3c852c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:16.806806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1f1fd-67c2-4212-8205-13aa3c852c33 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:16.834557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1318/2617] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:22:16 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:22:16.839374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98481f63-dd03-4abe-a7e6-15288f634c5a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:16.851620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98481f63-dd03-4abe-a7e6-15288f634c5a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:16.878418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:16.910495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b8fcc-cdfe-4723-86ae-cec283db36ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:16.911015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b8fcc-cdfe-4723-86ae-cec283db36ec admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:16.911291 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b8fcc-cdfe-4723-86ae-cec283db36ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:16.911556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471b8fcc-cdfe-4723-86ae-cec283db36ec admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:16.912174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b8fcc-cdfe-4723-86ae-cec283db36ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:16.914145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471b8fcc-cdfe-4723-86ae-cec283db36ec admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:16.935775 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1300/2618] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:22:16 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:16.939916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1f1fd-67c2-4212-8205-13aa3c852c33 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:16.950263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1f1fd-67c2-4212-8205-13aa3c852c33 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:16.954814 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d02d2f-07bf-440b-880e-14bf416feb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:16.955396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d02d2f-07bf-440b-880e-14bf416feb73 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:16.955668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d02d2f-07bf-440b-880e-14bf416feb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:16.955934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d02d2f-07bf-440b-880e-14bf416feb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:16.956563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d02d2f-07bf-440b-880e-14bf416feb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:16.958168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d02d2f-07bf-440b-880e-14bf416feb73 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:16.968433 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1319/2619] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:16.972233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b8fcc-cdfe-4723-86ae-cec283db36ec admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:16.983285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471b8fcc-cdfe-4723-86ae-cec283db36ec admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:16.987973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24f994d-b877-4493-825f-10c310a1b3c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:16.988562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f994d-b877-4493-825f-10c310a1b3c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:16.988843 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f994d-b877-4493-825f-10c310a1b3c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:16.989141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24f994d-b877-4493-825f-10c310a1b3c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:16.989789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f994d-b877-4493-825f-10c310a1b3c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:16.991369 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24f994d-b877-4493-825f-10c310a1b3c6 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.001566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1301/2620] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:17.005827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d02d2f-07bf-440b-880e-14bf416feb73 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.015710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d02d2f-07bf-440b-880e-14bf416feb73 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.020211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d3a94-e2de-4280-87cd-25744c183616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.020839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d3a94-e2de-4280-87cd-25744c183616 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.021134 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d3a94-e2de-4280-87cd-25744c183616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.021404 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010d3a94-e2de-4280-87cd-25744c183616 admin admin] PATH_INFO: `/v3/projects/ba21fedd44cd4dddad7f5ecc4ab77637/users/56e8d125813040b3887e24e98add0b5b/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.036085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d3a94-e2de-4280-87cd-25744c183616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba21fedd44cd4dddad7f5ecc4ab77637, user_id=56e8d125813040b3887e24e98add0b5b, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.038710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010d3a94-e2de-4280-87cd-25744c183616 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.052536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1320/2621] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:17 2022] PUT /identity/v3/projects/ba21fedd44cd4dddad7f5ecc4ab77637/users/56e8d125813040b3887e24e98add0b5b/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:22:17.065266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbc15d-bd3a-4d0b-8cdc-dd4c3fdbce90 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.065266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbc15d-bd3a-4d0b-8cdc-dd4c3fdbce90 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.065266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdbc15d-bd3a-4d0b-8cdc-dd4c3fdbce90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.098815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1cdbc15d-bd3a-4d0b-8cdc-dd4c3fdbce90 None None] MFA Rules not processed for user `56e8d125813040b3887e24e98add0b5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:17.100032 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1cdbc15d-bd3a-4d0b-8cdc-dd4c3fdbce90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:17.165677 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1302/2622] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:17.170613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d3a94-e2de-4280-87cd-25744c183616 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.231747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010d3a94-e2de-4280-87cd-25744c183616 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.236573 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48b9cd-979c-4d2e-8584-6a0ae10eab1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.237205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48b9cd-979c-4d2e-8584-6a0ae10eab1c admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.237479 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48b9cd-979c-4d2e-8584-6a0ae10eab1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.237745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48b9cd-979c-4d2e-8584-6a0ae10eab1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.238378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48b9cd-979c-4d2e-8584-6a0ae10eab1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.240240 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c48b9cd-979c-4d2e-8584-6a0ae10eab1c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.268079 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1321/2623] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:22:17 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:17.272459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbc15d-bd3a-4d0b-8cdc-dd4c3fdbce90 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.282925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdbc15d-bd3a-4d0b-8cdc-dd4c3fdbce90 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.337978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713808d-27a2-4150-8cdd-56129f04b6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.338572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713808d-27a2-4150-8cdd-56129f04b6f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.338845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713808d-27a2-4150-8cdd-56129f04b6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.339239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713808d-27a2-4150-8cdd-56129f04b6f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.339853 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713808d-27a2-4150-8cdd-56129f04b6f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.345307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713808d-27a2-4150-8cdd-56129f04b6f2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.367539 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1303/2624] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:22:17 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:17.373613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48b9cd-979c-4d2e-8584-6a0ae10eab1c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.385047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48b9cd-979c-4d2e-8584-6a0ae10eab1c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.390106 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eae618-34fe-4f26-8b28-98ea10a07354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.390696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eae618-34fe-4f26-8b28-98ea10a07354 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.390970 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eae618-34fe-4f26-8b28-98ea10a07354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.391236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eae618-34fe-4f26-8b28-98ea10a07354 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.391869 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eae618-34fe-4f26-8b28-98ea10a07354 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.393471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eae618-34fe-4f26-8b28-98ea10a07354 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.404137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1322/2625] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:17.408114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713808d-27a2-4150-8cdd-56129f04b6f2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.418864 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713808d-27a2-4150-8cdd-56129f04b6f2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.423441 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6707aa3-efc7-4daf-bb4b-262f0986caf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.424026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707aa3-efc7-4daf-bb4b-262f0986caf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.424299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707aa3-efc7-4daf-bb4b-262f0986caf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.424567 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6707aa3-efc7-4daf-bb4b-262f0986caf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.425233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6707aa3-efc7-4daf-bb4b-262f0986caf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.426797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6707aa3-efc7-4daf-bb4b-262f0986caf9 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.437363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1304/2626] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:17.441361 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eae618-34fe-4f26-8b28-98ea10a07354 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.451482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eae618-34fe-4f26-8b28-98ea10a07354 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.456117 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d4047-5396-4125-9774-ee1f560f3ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.456750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4047-5396-4125-9774-ee1f560f3ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.457030 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4047-5396-4125-9774-ee1f560f3ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.457349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797d4047-5396-4125-9774-ee1f560f3ec0 admin admin] PATH_INFO: `/v3/projects/312dc416305440f180110217b8c4e737/users/8154f39dadf8418f870856a37a979bb9/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.471486 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d4047-5396-4125-9774-ee1f560f3ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=312dc416305440f180110217b8c4e737, user_id=8154f39dadf8418f870856a37a979bb9, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.474114 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797d4047-5396-4125-9774-ee1f560f3ec0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.483688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:17.489498 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1323/2627] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:17 2022] PUT /identity/v3/projects/312dc416305440f180110217b8c4e737/users/8154f39dadf8418f870856a37a979bb9/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:17.500851 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b15898-4b30-4644-8305-5a54c54a661b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.501158 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b15898-4b30-4644-8305-5a54c54a661b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.501433 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b15898-4b30-4644-8305-5a54c54a661b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.530758 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c7b15898-4b30-4644-8305-5a54c54a661b None None] MFA Rules not processed for user `8154f39dadf8418f870856a37a979bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:17.531964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c7b15898-4b30-4644-8305-5a54c54a661b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:17.535916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:17.599127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1305/2628] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:17.603743 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d4047-5396-4125-9774-ee1f560f3ec0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.665471 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797d4047-5396-4125-9774-ee1f560f3ec0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.670902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fa879-3139-462e-8430-f74d5c3637b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.671438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fa879-3139-462e-8430-f74d5c3637b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.671714 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fa879-3139-462e-8430-f74d5c3637b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.671983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1fa879-3139-462e-8430-f74d5c3637b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.672620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1fa879-3139-462e-8430-f74d5c3637b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.674527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1fa879-3139-462e-8430-f74d5c3637b3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.700248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1324/2629] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:22:17 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:22:17.704419 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b15898-4b30-4644-8305-5a54c54a661b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.715492 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b15898-4b30-4644-8305-5a54c54a661b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.770629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb2f8f-296d-4278-b48f-d4f245465cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.771247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb2f8f-296d-4278-b48f-d4f245465cab admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.771528 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb2f8f-296d-4278-b48f-d4f245465cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.772419 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deb2f8f-296d-4278-b48f-d4f245465cab admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.773069 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb2f8f-296d-4278-b48f-d4f245465cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.775003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deb2f8f-296d-4278-b48f-d4f245465cab admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.797288 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1306/2630] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:22:17 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:17.801304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fa879-3139-462e-8430-f74d5c3637b3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.811957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1fa879-3139-462e-8430-f74d5c3637b3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.816588 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a47ad9-5ae6-44d7-906b-4a4e9a85034c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.817210 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47ad9-5ae6-44d7-906b-4a4e9a85034c admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.817490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47ad9-5ae6-44d7-906b-4a4e9a85034c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.817764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a47ad9-5ae6-44d7-906b-4a4e9a85034c admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.818403 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a47ad9-5ae6-44d7-906b-4a4e9a85034c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.819969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a47ad9-5ae6-44d7-906b-4a4e9a85034c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.830357 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1325/2631] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:17.834644 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb2f8f-296d-4278-b48f-d4f245465cab admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.846919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deb2f8f-296d-4278-b48f-d4f245465cab admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.851553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b266272-a724-4fdb-9408-9bc317024c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.852196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266272-a724-4fdb-9408-9bc317024c1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.852474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266272-a724-4fdb-9408-9bc317024c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.852747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b266272-a724-4fdb-9408-9bc317024c1a admin admin] PATH_INFO: `/v3/projects/78e34e1323b642ac93395b3674f6e302/users/b7fa72b74f2b493ca0af72794a415f55/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.868144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b266272-a724-4fdb-9408-9bc317024c1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e34e1323b642ac93395b3674f6e302, user_id=b7fa72b74f2b493ca0af72794a415f55, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.870934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b266272-a724-4fdb-9408-9bc317024c1a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.883598 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1307/2632] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:17 2022] PUT /identity/v3/projects/78e34e1323b642ac93395b3674f6e302/users/b7fa72b74f2b493ca0af72794a415f55/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:17.887998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a47ad9-5ae6-44d7-906b-4a4e9a85034c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.898108 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a47ad9-5ae6-44d7-906b-4a4e9a85034c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.954177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6565536b-d605-4a20-a8f5-7cd59c004d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.954704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565536b-d605-4a20-a8f5-7cd59c004d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.954983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565536b-d605-4a20-a8f5-7cd59c004d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.955261 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565536b-d605-4a20-a8f5-7cd59c004d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.955891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565536b-d605-4a20-a8f5-7cd59c004d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.957493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6565536b-d605-4a20-a8f5-7cd59c004d41 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:17.967612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1326/2633] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:17.972695 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b266272-a724-4fdb-9408-9bc317024c1a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:17.983140 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b266272-a724-4fdb-9408-9bc317024c1a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:17.987820 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c88b2e-428f-4a62-9c7f-8cf8830cf42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:17.988457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c88b2e-428f-4a62-9c7f-8cf8830cf42b admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:17.988734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c88b2e-428f-4a62-9c7f-8cf8830cf42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:17.989006 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c88b2e-428f-4a62-9c7f-8cf8830cf42b admin admin] PATH_INFO: `/v3/projects/78e34e1323b642ac93395b3674f6e302/users/b7fa72b74f2b493ca0af72794a415f55/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:17.990980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c88b2e-428f-4a62-9c7f-8cf8830cf42b admin admin] RBAC: Authorizing `identity:create_grant(project_id=78e34e1323b642ac93395b3674f6e302, user_id=b7fa72b74f2b493ca0af72794a415f55, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:17.993649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c88b2e-428f-4a62-9c7f-8cf8830cf42b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:18.003642 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1308/2634] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:17 2022] PUT /identity/v3/projects/78e34e1323b642ac93395b3674f6e302/users/b7fa72b74f2b493ca0af72794a415f55/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:18.015335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d71d0-73e6-4c7b-a410-fcb4453cd86e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:18.015623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d71d0-73e6-4c7b-a410-fcb4453cd86e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:18.015901 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1d71d0-73e6-4c7b-a410-fcb4453cd86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:18.045358 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8f1d71d0-73e6-4c7b-a410-fcb4453cd86e None None] MFA Rules not processed for user `b7fa72b74f2b493ca0af72794a415f55`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:18.046570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8f1d71d0-73e6-4c7b-a410-fcb4453cd86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:18.104902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:18.114288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1327/2635] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:18.125134 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c88b2e-428f-4a62-9c7f-8cf8830cf42b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:18.126902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-36c88b2e-428f-4a62-9c7f-8cf8830cf42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:18.183494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c88b2e-428f-4a62-9c7f-8cf8830cf42b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:18.187372 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71400a81-4f02-4ecf-b551-e290606876ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:18.187964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71400a81-4f02-4ecf-b551-e290606876ba service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:18.188240 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71400a81-4f02-4ecf-b551-e290606876ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:18.188510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71400a81-4f02-4ecf-b551-e290606876ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:18.197327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71400a81-4f02-4ecf-b551-e290606876ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:18.199443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71400a81-4f02-4ecf-b551-e290606876ba service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:18.204527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:18.209779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1309/2636] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:18.712540 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d71d0-73e6-4c7b-a410-fcb4453cd86e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:18.722223 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1d71d0-73e6-4c7b-a410-fcb4453cd86e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:18.725795 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b529f1-13fb-47aa-b354-95d32a1f8b5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:18.726388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b529f1-13fb-47aa-b354-95d32a1f8b5c service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:18.726668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b529f1-13fb-47aa-b354-95d32a1f8b5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:18.726937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b529f1-13fb-47aa-b354-95d32a1f8b5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:18.782797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b529f1-13fb-47aa-b354-95d32a1f8b5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:18.784225 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b529f1-13fb-47aa-b354-95d32a1f8b5c service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:18.793618 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1328/2637] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:24.927405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71400a81-4f02-4ecf-b551-e290606876ba service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:24.928963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-71400a81-4f02-4ecf-b551-e290606876ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:24.940379 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71400a81-4f02-4ecf-b551-e290606876ba service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:24.947333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb46ad6-90d7-4a33-bc45-551f30f65084 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:24.947958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb46ad6-90d7-4a33-bc45-551f30f65084 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:24.948230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb46ad6-90d7-4a33-bc45-551f30f65084 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:24.948495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb46ad6-90d7-4a33-bc45-551f30f65084 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:25.004983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb46ad6-90d7-4a33-bc45-551f30f65084 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:25.006450 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb46ad6-90d7-4a33-bc45-551f30f65084 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:25.015905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1310/2638] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:27.087196 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b529f1-13fb-47aa-b354-95d32a1f8b5c service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:27.147866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b529f1-13fb-47aa-b354-95d32a1f8b5c service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:27.152371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:27.153676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:27.153952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:27.154319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] PATH_INFO: `/v3/users/56e8d125813040b3887e24e98add0b5b` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:27.155440 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56e8d125813040b3887e24e98add0b5b)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:27.157348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:27.204425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56e8d125813040b3887e24e98add0b5b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.205165 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56e8d125813040b3887e24e98add0b5b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.205840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56e8d125813040b3887e24e98add0b5b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.218478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56e8d125813040b3887e24e98add0b5b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.225128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56e8d125813040b3887e24e98add0b5b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.238095 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1329/2639] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:27 2022] DELETE /identity/v3/users/56e8d125813040b3887e24e98add0b5b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:27.242603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc631e-cbc3-4a97-92c5-c61338cd7554 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:27.243112 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc631e-cbc3-4a97-92c5-c61338cd7554 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:27.243224 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc631e-cbc3-4a97-92c5-c61338cd7554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:27.261757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-f2bc631e-cbc3-4a97-92c5-c61338cd7554 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:27.263700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f2bc631e-cbc3-4a97-92c5-c61338cd7554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:27.321245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1311/2640] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:27.331476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:27.333063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:27.347493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:27.392131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38185277-6a81-42d7-be66-a10602da9f54 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:27.395724 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617d94c-9289-49a2-a59c-eea6f28555ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:27.396316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617d94c-9289-49a2-a59c-eea6f28555ad service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:27.396590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617d94c-9289-49a2-a59c-eea6f28555ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:27.396861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617d94c-9289-49a2-a59c-eea6f28555ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:27.405785 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617d94c-9289-49a2-a59c-eea6f28555ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:27.407194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617d94c-9289-49a2-a59c-eea6f28555ad service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:27.416437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1330/2641] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:22:27.694836 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc631e-cbc3-4a97-92c5-c61338cd7554 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:27.696332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f2bc631e-cbc3-4a97-92c5-c61338cd7554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:27.710683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc631e-cbc3-4a97-92c5-c61338cd7554 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:27.716459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:27.717138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:27.717396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:27.717725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] PATH_INFO: `/v3/projects/ba21fedd44cd4dddad7f5ecc4ab77637` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:27.718979 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba21fedd44cd4dddad7f5ecc4ab77637)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:27.720879 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:27.767913 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba21fedd44cd4dddad7f5ecc4ab77637', 'request_id': 'req-320e3571-f017-4970-8642-5b520bb730b7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.774692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba21fedd44cd4dddad7f5ecc4ab77637', 'request_id': 'req-320e3571-f017-4970-8642-5b520bb730b7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.781293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba21fedd44cd4dddad7f5ecc4ab77637', 'request_id': 'req-320e3571-f017-4970-8642-5b520bb730b7'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.791807 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] The token cache is being invalidate because project ba21fedd44cd4dddad7f5ecc4ab77637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:27.792094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.792752 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.794930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1312/2642] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:27 2022] DELETE /identity/v3/projects/ba21fedd44cd4dddad7f5ecc4ab77637 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:27.798861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617d94c-9289-49a2-a59c-eea6f28555ad service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:27.808491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617d94c-9289-49a2-a59c-eea6f28555ad service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:27.865503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:27.866125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:27.866401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:27.866678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] PATH_INFO: `/v3/users/8154f39dadf8418f870856a37a979bb9` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:27.867778 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8154f39dadf8418f870856a37a979bb9)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:27.869705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:27.913386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8154f39dadf8418f870856a37a979bb9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.914040 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8154f39dadf8418f870856a37a979bb9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.914635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8154f39dadf8418f870856a37a979bb9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.926753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8154f39dadf8418f870856a37a979bb9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.933514 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8154f39dadf8418f870856a37a979bb9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:27.946800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1331/2643] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:27 2022] DELETE /identity/v3/users/8154f39dadf8418f870856a37a979bb9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:28.242406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:28.244114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:28.266821 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e3571-f017-4970-8642-5b520bb730b7 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:28.290588 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:28.334111 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:28.334301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:28.334556 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:28.334827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] PATH_INFO: `/v3/projects/312dc416305440f180110217b8c4e737` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:28.335965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=312dc416305440f180110217b8c4e737)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:28.337906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:28.387535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '312dc416305440f180110217b8c4e737', 'request_id': 'req-d01502ea-dece-44fc-abcf-35c84998c4ae'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.394582 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '312dc416305440f180110217b8c4e737', 'request_id': 'req-d01502ea-dece-44fc-abcf-35c84998c4ae'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.401517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '312dc416305440f180110217b8c4e737', 'request_id': 'req-d01502ea-dece-44fc-abcf-35c84998c4ae'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.414860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] The token cache is being invalidate because project 312dc416305440f180110217b8c4e737 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:28.415368 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.416020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.417711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1313/2644] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:28 2022] DELETE /identity/v3/projects/312dc416305440f180110217b8c4e737 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:28.422550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:28.424251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:28.437766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988cdde5-c2ab-4cc7-bd18-dac3a01bfa7c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:28.473006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:28.500032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6637b60-96b3-4ea9-949b-49e5ea2e2ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:28.500661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6637b60-96b3-4ea9-949b-49e5ea2e2ca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:28.500934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6637b60-96b3-4ea9-949b-49e5ea2e2ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:28.501241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6637b60-96b3-4ea9-949b-49e5ea2e2ca8 admin admin] PATH_INFO: `/v3/users/b7fa72b74f2b493ca0af72794a415f55` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:28.502354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6637b60-96b3-4ea9-949b-49e5ea2e2ca8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7fa72b74f2b493ca0af72794a415f55)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:28.504244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6637b60-96b3-4ea9-949b-49e5ea2e2ca8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:28.549533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d6637b60-96b3-4ea9-949b-49e5ea2e2ca8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7fa72b74f2b493ca0af72794a415f55'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.550121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d6637b60-96b3-4ea9-949b-49e5ea2e2ca8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7fa72b74f2b493ca0af72794a415f55'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.550721 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d6637b60-96b3-4ea9-949b-49e5ea2e2ca8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7fa72b74f2b493ca0af72794a415f55'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.562801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d6637b60-96b3-4ea9-949b-49e5ea2e2ca8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7fa72b74f2b493ca0af72794a415f55'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.569553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-d6637b60-96b3-4ea9-949b-49e5ea2e2ca8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7fa72b74f2b493ca0af72794a415f55'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.582410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1332/2645] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:28 2022] DELETE /identity/v3/users/b7fa72b74f2b493ca0af72794a415f55 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:22:28.860824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:28.862367 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:28.875712 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01502ea-dece-44fc-abcf-35c84998c4ae admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:28.933422 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:28.934048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:28.934326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:28.935104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] PATH_INFO: `/v3/projects/78e34e1323b642ac93395b3674f6e302` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:28.936587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78e34e1323b642ac93395b3674f6e302)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:28.938574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:28.984532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78e34e1323b642ac93395b3674f6e302', 'request_id': 'req-38b9125c-a3a7-413f-949e-50556c69b378'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.988066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:28.991611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78e34e1323b642ac93395b3674f6e302', 'request_id': 'req-38b9125c-a3a7-413f-949e-50556c69b378'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:28.999657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78e34e1323b642ac93395b3674f6e302', 'request_id': 'req-38b9125c-a3a7-413f-949e-50556c69b378'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:29.011126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] The token cache is being invalidate because project 78e34e1323b642ac93395b3674f6e302 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:29.011414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:29.012055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-38b9125c-a3a7-413f-949e-50556c69b378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:29.014188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1314/2646] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:28 2022] DELETE /identity/v3/projects/78e34e1323b642ac93395b3674f6e302 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:29.020112 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f94e3-7213-4677-b21a-c93be5d10f0f None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.020428 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f94e3-7213-4677-b21a-c93be5d10f0f None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.020709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f94e3-7213-4677-b21a-c93be5d10f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.039882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3f8f94e3-7213-4677-b21a-c93be5d10f0f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:29.041081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3f8f94e3-7213-4677-b21a-c93be5d10f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:29.100484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1333/2647] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:29.109177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8901b07-13f5-414f-8ca9-b70449a20c62 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.109420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8901b07-13f5-414f-8ca9-b70449a20c62 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.109696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8901b07-13f5-414f-8ca9-b70449a20c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.129444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c8901b07-13f5-414f-8ca9-b70449a20c62 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:29.130641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c8901b07-13f5-414f-8ca9-b70449a20c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:29.139239 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1315/2648] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:29.143320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f94e3-7213-4677-b21a-c93be5d10f0f None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:29.152818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f94e3-7213-4677-b21a-c93be5d10f0f None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:29.158840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c514-0f8c-41ea-a7a1-62925621a125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:29.159447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c514-0f8c-41ea-a7a1-62925621a125 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.159730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c514-0f8c-41ea-a7a1-62925621a125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.160000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd52c514-0f8c-41ea-a7a1-62925621a125 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.160641 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52c514-0f8c-41ea-a7a1-62925621a125 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:22:29.160944 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52c514-0f8c-41ea-a7a1-62925621a125 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:29.162550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd52c514-0f8c-41ea-a7a1-62925621a125 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:29.174580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1334/2649] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:22:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:22:29.178895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8901b07-13f5-414f-8ca9-b70449a20c62 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:29.188791 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8901b07-13f5-414f-8ca9-b70449a20c62 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:29.193437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e84e01-84ef-41d9-bcb3-771d85f9b8ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:29.194020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e84e01-84ef-41d9-bcb3-771d85f9b8ce admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.194296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e84e01-84ef-41d9-bcb3-771d85f9b8ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.194562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e84e01-84ef-41d9-bcb3-771d85f9b8ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.195177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e84e01-84ef-41d9-bcb3-771d85f9b8ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:29.197050 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e84e01-84ef-41d9-bcb3-771d85f9b8ce admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:29.224099 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1316/2650] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:22:29 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:29.228311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c514-0f8c-41ea-a7a1-62925621a125 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:29.239031 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd52c514-0f8c-41ea-a7a1-62925621a125 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:29.279734 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:29.297262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a23c6-b445-4b9e-8f94-df953ce7cfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:29.297865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a23c6-b445-4b9e-8f94-df953ce7cfdb admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.298141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a23c6-b445-4b9e-8f94-df953ce7cfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.298410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a23c6-b445-4b9e-8f94-df953ce7cfdb admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.299048 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a23c6-b445-4b9e-8f94-df953ce7cfdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:29.300924 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a23c6-b445-4b9e-8f94-df953ce7cfdb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:29.322005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1335/2651] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:22:29 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:29.326208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e84e01-84ef-41d9-bcb3-771d85f9b8ce admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:29.336465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e84e01-84ef-41d9-bcb3-771d85f9b8ce admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:29.341029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95729d0b-332b-49b3-bc38-d66459f08ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:29.341650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95729d0b-332b-49b3-bc38-d66459f08ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.341929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95729d0b-332b-49b3-bc38-d66459f08ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.342203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95729d0b-332b-49b3-bc38-d66459f08ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.342863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95729d0b-332b-49b3-bc38-d66459f08ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:29.344446 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95729d0b-332b-49b3-bc38-d66459f08ee7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:29.354217 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1317/2652] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:22:29.358530 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a23c6-b445-4b9e-8f94-df953ce7cfdb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:29.368213 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a23c6-b445-4b9e-8f94-df953ce7cfdb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:29.373674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c03291-d9ed-4885-a7c0-490e14a0cd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:29.374269 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c03291-d9ed-4885-a7c0-490e14a0cd5b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.374548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c03291-d9ed-4885-a7c0-490e14a0cd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.374821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c03291-d9ed-4885-a7c0-490e14a0cd5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.375490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c03291-d9ed-4885-a7c0-490e14a0cd5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:29.377053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c03291-d9ed-4885-a7c0-490e14a0cd5b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:29.387581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1336/2653] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:29.391535 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95729d0b-332b-49b3-bc38-d66459f08ee7 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:29.401429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95729d0b-332b-49b3-bc38-d66459f08ee7 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:29.406128 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b5c3f-b69c-41c3-85e9-60612644de88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:29.406758 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b5c3f-b69c-41c3-85e9-60612644de88 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.407033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b5c3f-b69c-41c3-85e9-60612644de88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.407301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b5c3f-b69c-41c3-85e9-60612644de88 admin admin] PATH_INFO: `/v3/projects/f98adda9ad9c4348ab461e119c5ed82d/users/c002434c953045fab336fc01eeb72767/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.422047 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b5c3f-b69c-41c3-85e9-60612644de88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f98adda9ad9c4348ab461e119c5ed82d, user_id=c002434c953045fab336fc01eeb72767, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:29.424668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427b5c3f-b69c-41c3-85e9-60612644de88 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:29.437706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1318/2654] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:29 2022] PUT /identity/v3/projects/f98adda9ad9c4348ab461e119c5ed82d/users/c002434c953045fab336fc01eeb72767/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:29.505912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd92ef4-3c53-4761-8330-7d3101e5b117 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.506902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd92ef4-3c53-4761-8330-7d3101e5b117 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.507395 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd92ef4-3c53-4761-8330-7d3101e5b117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.507574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b5c3f-b69c-41c3-85e9-60612644de88 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:29.509374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-427b5c3f-b69c-41c3-85e9-60612644de88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:29.535994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3bd92ef4-3c53-4761-8330-7d3101e5b117 None None] MFA Rules not processed for user `c002434c953045fab336fc01eeb72767`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:29.537237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3bd92ef4-3c53-4761-8330-7d3101e5b117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:29.574069 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b5c3f-b69c-41c3-85e9-60612644de88 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:29.579660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37a147d-d24c-44a7-b2d5-e84885d35b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:29.580287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a147d-d24c-44a7-b2d5-e84885d35b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.580570 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a147d-d24c-44a7-b2d5-e84885d35b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.580850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37a147d-d24c-44a7-b2d5-e84885d35b43 admin admin] PATH_INFO: `/v3/users/dc981727c70346c983f532920541cc9b` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.582024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a147d-d24c-44a7-b2d5-e84885d35b43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc981727c70346c983f532920541cc9b)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:29.583971 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37a147d-d24c-44a7-b2d5-e84885d35b43 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:29.600950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1337/2655] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:29.612315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd92ef4-3c53-4761-8330-7d3101e5b117 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:29.613843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3bd92ef4-3c53-4761-8330-7d3101e5b117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:29.632245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c37a147d-d24c-44a7-b2d5-e84885d35b43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc981727c70346c983f532920541cc9b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:29.632874 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c37a147d-d24c-44a7-b2d5-e84885d35b43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc981727c70346c983f532920541cc9b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:29.640250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c37a147d-d24c-44a7-b2d5-e84885d35b43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc981727c70346c983f532920541cc9b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:29.640920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c37a147d-d24c-44a7-b2d5-e84885d35b43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc981727c70346c983f532920541cc9b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:29.652769 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-c37a147d-d24c-44a7-b2d5-e84885d35b43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc981727c70346c983f532920541cc9b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:29.661167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:29.666578 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1319/2656] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:29 2022] DELETE /identity/v3/users/dc981727c70346c983f532920541cc9b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:22:29.671223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f4947-4dbc-4ca3-baf4-ec54639b5c28 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.671798 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f4947-4dbc-4ca3-baf4-ec54639b5c28 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.672235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f4947-4dbc-4ca3-baf4-ec54639b5c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.673940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd92ef4-3c53-4761-8330-7d3101e5b117 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:29.692167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-834f4947-4dbc-4ca3-baf4-ec54639b5c28 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:29.693672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-834f4947-4dbc-4ca3-baf4-ec54639b5c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:29.725776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b970e-aa2b-4621-aab7-e29e7480aaae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:29.726386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b970e-aa2b-4621-aab7-e29e7480aaae service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.726661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b970e-aa2b-4621-aab7-e29e7480aaae service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.726928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b970e-aa2b-4621-aab7-e29e7480aaae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.728679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-947b970e-aa2b-4621-aab7-e29e7480aaae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:29.749369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:29.753561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1320/2657] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:22:29.763910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f4947-4dbc-4ca3-baf4-ec54639b5c28 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:29.765345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-834f4947-4dbc-4ca3-baf4-ec54639b5c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:29.790179 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b970e-aa2b-4621-aab7-e29e7480aaae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:29.791596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947b970e-aa2b-4621-aab7-e29e7480aaae service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:29.800666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1338/2658] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:22:29.823064 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f4947-4dbc-4ca3-baf4-ec54639b5c28 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:29.826941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a195bb-10fe-4fc4-a056-126377add191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:29.827538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a195bb-10fe-4fc4-a056-126377add191 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:29.827818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a195bb-10fe-4fc4-a056-126377add191 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:29.828090 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a195bb-10fe-4fc4-a056-126377add191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:29.837305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a195bb-10fe-4fc4-a056-126377add191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:29.838713 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a195bb-10fe-4fc4-a056-126377add191 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:29.851222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1321/2659] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:22:30.179627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b970e-aa2b-4621-aab7-e29e7480aaae service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:30.181135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-947b970e-aa2b-4621-aab7-e29e7480aaae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:30.198001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b970e-aa2b-4621-aab7-e29e7480aaae service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:30.205193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:30.205844 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:30.206154 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:30.206468 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] PATH_INFO: `/v3/projects/e52d267b52bd4f8ca0e7ea9ab2b4c9d5` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:30.207562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e52d267b52bd4f8ca0e7ea9ab2b4c9d5)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:30.209564 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:30.257128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e52d267b52bd4f8ca0e7ea9ab2b4c9d5', 'request_id': 'req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:30.263887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e52d267b52bd4f8ca0e7ea9ab2b4c9d5', 'request_id': 'req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:30.269687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e52d267b52bd4f8ca0e7ea9ab2b4c9d5', 'request_id': 'req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:30.282682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] The token cache is being invalidate because project e52d267b52bd4f8ca0e7ea9ab2b4c9d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:30.282983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:30.283613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-446615d5-4c8e-4c0b-aefb-c53b56fca4cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:30.285085 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1339/2660] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:30 2022] DELETE /identity/v3/projects/e52d267b52bd4f8ca0e7ea9ab2b4c9d5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:30.289988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c0f7e-2c0e-4039-ac76-eed2b4956d03 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:30.290274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c0f7e-2c0e-4039-ac76-eed2b4956d03 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:30.290542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c0f7e-2c0e-4039-ac76-eed2b4956d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:30.310095 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-268c0f7e-2c0e-4039-ac76-eed2b4956d03 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:30.311547 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-268c0f7e-2c0e-4039-ac76-eed2b4956d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:30.369758 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1322/2661] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:22:30.379808 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68675-6de9-4103-9062-b28e59571d1c None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:30.380095 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68675-6de9-4103-9062-b28e59571d1c None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:30.380366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b68675-6de9-4103-9062-b28e59571d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:30.400704 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a4b68675-6de9-4103-9062-b28e59571d1c None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:30.401891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a4b68675-6de9-4103-9062-b28e59571d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:30.409800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1340/2662] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:30.413984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c0f7e-2c0e-4039-ac76-eed2b4956d03 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:30.423569 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c0f7e-2c0e-4039-ac76-eed2b4956d03 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:30.428126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942b88f-2db3-467b-9ecc-237983808b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:30.428690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942b88f-2db3-467b-9ecc-237983808b59 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:30.428961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942b88f-2db3-467b-9ecc-237983808b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:30.429270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942b88f-2db3-467b-9ecc-237983808b59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:30.429901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942b88f-2db3-467b-9ecc-237983808b59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:22:30.430201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942b88f-2db3-467b-9ecc-237983808b59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:30.431773 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942b88f-2db3-467b-9ecc-237983808b59 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:30.443885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1323/2663] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:22:30.448157 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b68675-6de9-4103-9062-b28e59571d1c None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:30.458089 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b68675-6de9-4103-9062-b28e59571d1c None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:30.462835 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83308a87-45ff-4109-8b47-db3d2dd5fb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:30.463423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83308a87-45ff-4109-8b47-db3d2dd5fb26 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:30.463703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83308a87-45ff-4109-8b47-db3d2dd5fb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:30.463974 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83308a87-45ff-4109-8b47-db3d2dd5fb26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:30.464613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83308a87-45ff-4109-8b47-db3d2dd5fb26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:30.466792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83308a87-45ff-4109-8b47-db3d2dd5fb26 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:30.493950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1341/2664] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:22:30 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:30.498475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942b88f-2db3-467b-9ecc-237983808b59 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:30.508225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942b88f-2db3-467b-9ecc-237983808b59 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:30.534682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:30.567572 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667db77f-f852-4c49-8f8d-2d3daf518d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:30.568167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667db77f-f852-4c49-8f8d-2d3daf518d17 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:30.568444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667db77f-f852-4c49-8f8d-2d3daf518d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:30.568718 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667db77f-f852-4c49-8f8d-2d3daf518d17 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:30.569386 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667db77f-f852-4c49-8f8d-2d3daf518d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:30.571280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667db77f-f852-4c49-8f8d-2d3daf518d17 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:30.592373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1324/2665] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:22:30 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:22:30.596463 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83308a87-45ff-4109-8b47-db3d2dd5fb26 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:30.606706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83308a87-45ff-4109-8b47-db3d2dd5fb26 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:30.611470 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b4519-f8fb-470d-9279-0acfc14f6be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:30.612067 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b4519-f8fb-470d-9279-0acfc14f6be3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:30.612347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b4519-f8fb-470d-9279-0acfc14f6be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:30.612621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47b4519-f8fb-470d-9279-0acfc14f6be3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:30.613302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b4519-f8fb-470d-9279-0acfc14f6be3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:30.614882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47b4519-f8fb-470d-9279-0acfc14f6be3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:30.625245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1342/2666] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:30.628957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667db77f-f852-4c49-8f8d-2d3daf518d17 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:30.639502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667db77f-f852-4c49-8f8d-2d3daf518d17 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:30.643984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb501743-6141-49f4-abb8-703369cd8688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:30.644567 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb501743-6141-49f4-abb8-703369cd8688 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:30.644842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb501743-6141-49f4-abb8-703369cd8688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:30.645139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb501743-6141-49f4-abb8-703369cd8688 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:30.645764 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb501743-6141-49f4-abb8-703369cd8688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:30.647317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb501743-6141-49f4-abb8-703369cd8688 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:30.657749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1325/2667] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:30.661730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b4519-f8fb-470d-9279-0acfc14f6be3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:30.672188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47b4519-f8fb-470d-9279-0acfc14f6be3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:30.676689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafcbc5-dc46-4347-a527-9d6e0864661b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:30.677353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafcbc5-dc46-4347-a527-9d6e0864661b admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:30.677628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafcbc5-dc46-4347-a527-9d6e0864661b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:30.677897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faafcbc5-dc46-4347-a527-9d6e0864661b admin admin] PATH_INFO: `/v3/projects/ed0761f1f2a148ce8b55f519a13421c8/users/59d6a1d6b9bc4daf8542b58cf62fff1a/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:30.692945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafcbc5-dc46-4347-a527-9d6e0864661b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0761f1f2a148ce8b55f519a13421c8, user_id=59d6a1d6b9bc4daf8542b58cf62fff1a, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:30.695554 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faafcbc5-dc46-4347-a527-9d6e0864661b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:30.708066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1343/2668] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:30 2022] PUT /identity/v3/projects/ed0761f1f2a148ce8b55f519a13421c8/users/59d6a1d6b9bc4daf8542b58cf62fff1a/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:30.712907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f893c7d-4198-4211-8545-eb3c399af8bd None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:30.713230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f893c7d-4198-4211-8545-eb3c399af8bd None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:30.713499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f893c7d-4198-4211-8545-eb3c399af8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:30.732352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-1f893c7d-4198-4211-8545-eb3c399af8bd None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:30.733575 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1f893c7d-4198-4211-8545-eb3c399af8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:30.790863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1326/2669] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:30.801715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafcbc5-dc46-4347-a527-9d6e0864661b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:30.803398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-faafcbc5-dc46-4347-a527-9d6e0864661b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:30.852640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:30.868235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faafcbc5-dc46-4347-a527-9d6e0864661b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:30.871940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc1598-b473-4a8d-914f-946242fa72e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:30.873201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc1598-b473-4a8d-914f-946242fa72e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:30.873479 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc1598-b473-4a8d-914f-946242fa72e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:30.873774 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc1598-b473-4a8d-914f-946242fa72e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:30.883097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc1598-b473-4a8d-914f-946242fa72e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:30.884539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc1598-b473-4a8d-914f-946242fa72e7 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:30.894942 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1344/2670] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:31.564085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc67652-d127-4f48-85a7-bc712c82a115 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:31.564934 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc67652-d127-4f48-85a7-bc712c82a115 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:31.565266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc67652-d127-4f48-85a7-bc712c82a115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:31.594276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3bc67652-d127-4f48-85a7-bc712c82a115 None None] MFA Rules not processed for user `59d6a1d6b9bc4daf8542b58cf62fff1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:31.595509 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3bc67652-d127-4f48-85a7-bc712c82a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:31.661935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1327/2671] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:31.673617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc1598-b473-4a8d-914f-946242fa72e7 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:31.675127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-eebc1598-b473-4a8d-914f-946242fa72e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:31.734129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc1598-b473-4a8d-914f-946242fa72e7 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:31.737639 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172e7c5-7fbc-4fd3-bed0-580146c5cb88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:31.738262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172e7c5-7fbc-4fd3-bed0-580146c5cb88 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:31.738516 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172e7c5-7fbc-4fd3-bed0-580146c5cb88 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:31.738847 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172e7c5-7fbc-4fd3-bed0-580146c5cb88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:31.747177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172e7c5-7fbc-4fd3-bed0-580146c5cb88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:31.749151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172e7c5-7fbc-4fd3-bed0-580146c5cb88 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:31.757736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1345/2672] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:37.326552 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17bc58-6d3f-40c1-b5e4-b5c16c78b329 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:37.327139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17bc58-6d3f-40c1-b5e4-b5c16c78b329 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:37.327139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17bc58-6d3f-40c1-b5e4-b5c16c78b329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:37.346605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0b17bc58-6d3f-40c1-b5e4-b5c16c78b329 None None] MFA Rules not processed for user `59d6a1d6b9bc4daf8542b58cf62fff1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:37.347803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0b17bc58-6d3f-40c1-b5e4-b5c16c78b329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:37.355170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1328/2673] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:40.388251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172e7c5-7fbc-4fd3-bed0-580146c5cb88 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:40.389746 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d172e7c5-7fbc-4fd3-bed0-580146c5cb88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:40.405122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172e7c5-7fbc-4fd3-bed0-580146c5cb88 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:40.410487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:40.411110 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:40.411386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:40.411654 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] PATH_INFO: `/v3/users/c002434c953045fab336fc01eeb72767` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:40.412787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c002434c953045fab336fc01eeb72767)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:40.414718 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:40.460284 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c002434c953045fab336fc01eeb72767'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:40.460922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c002434c953045fab336fc01eeb72767'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:40.461544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c002434c953045fab336fc01eeb72767'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:40.474587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c002434c953045fab336fc01eeb72767'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:40.481473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c002434c953045fab336fc01eeb72767'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:40.494518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1346/2674] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:40 2022] DELETE /identity/v3/users/c002434c953045fab336fc01eeb72767 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:40.499431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7e1c-6163-45f9-81bd-28d81d79a2eb None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:40.500171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7e1c-6163-45f9-81bd-28d81d79a2eb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:40.500171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7b7e1c-6163-45f9-81bd-28d81d79a2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:40.519075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-ed7b7e1c-6163-45f9-81bd-28d81d79a2eb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:40.520270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ed7b7e1c-6163-45f9-81bd-28d81d79a2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:40.546230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:40.590184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1329/2675] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:40.600598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:40.602277 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:40.666203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793c44fd-42c8-4cfe-86ad-1836cf1bb229 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:40.670010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40cbc2-903e-4004-8a03-fcdb47b2cf90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:40.670697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40cbc2-903e-4004-8a03-fcdb47b2cf90 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:40.671045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40cbc2-903e-4004-8a03-fcdb47b2cf90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:40.671384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40cbc2-903e-4004-8a03-fcdb47b2cf90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:40.681696 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40cbc2-903e-4004-8a03-fcdb47b2cf90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:40.683411 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f40cbc2-903e-4004-8a03-fcdb47b2cf90 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:40.690603 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:40.706755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1347/2676] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:41.146026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b7e1c-6163-45f9-81bd-28d81d79a2eb None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:41.147456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-ed7b7e1c-6163-45f9-81bd-28d81d79a2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:41.160685 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7b7e1c-6163-45f9-81bd-28d81d79a2eb None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:41.166624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:41.167484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:41.167766 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:41.168039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] PATH_INFO: `/v3/projects/f98adda9ad9c4348ab461e119c5ed82d` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:41.169293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f98adda9ad9c4348ab461e119c5ed82d)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:41.171197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:41.228332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f98adda9ad9c4348ab461e119c5ed82d', 'request_id': 'req-7441a505-5171-40d4-b3c0-8c7f1225d455'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:41.237026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f98adda9ad9c4348ab461e119c5ed82d', 'request_id': 'req-7441a505-5171-40d4-b3c0-8c7f1225d455'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:41.245193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f98adda9ad9c4348ab461e119c5ed82d', 'request_id': 'req-7441a505-5171-40d4-b3c0-8c7f1225d455'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:41.257978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] The token cache is being invalidate because project f98adda9ad9c4348ab461e119c5ed82d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:41.258416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:41.259229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-7441a505-5171-40d4-b3c0-8c7f1225d455 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:41.260814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1330/2677] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:41 2022] DELETE /identity/v3/projects/f98adda9ad9c4348ab461e119c5ed82d => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:41.266691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50063319-56a6-48c5-9a57-483176fad307 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:41.266966 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50063319-56a6-48c5-9a57-483176fad307 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:41.267237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50063319-56a6-48c5-9a57-483176fad307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:41.289562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-50063319-56a6-48c5-9a57-483176fad307 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:41.290983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-50063319-56a6-48c5-9a57-483176fad307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:41.357389 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1348/2678] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:41.365380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf4732-1001-473d-8240-c221f226a6f3 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:41.365671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf4732-1001-473d-8240-c221f226a6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:41.365963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cf4732-1001-473d-8240-c221f226a6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:41.385675 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-b0cf4732-1001-473d-8240-c221f226a6f3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:41.387000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-b0cf4732-1001-473d-8240-c221f226a6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:41.395633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1331/2679] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:41.399855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50063319-56a6-48c5-9a57-483176fad307 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:41.410906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50063319-56a6-48c5-9a57-483176fad307 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:41.415529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0d9c5-fd12-4930-a2fa-23416ad90c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:41.416147 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0d9c5-fd12-4930-a2fa-23416ad90c7d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:41.416422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0d9c5-fd12-4930-a2fa-23416ad90c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:41.416691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c0d9c5-fd12-4930-a2fa-23416ad90c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:41.417365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0d9c5-fd12-4930-a2fa-23416ad90c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:22:41.417665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0d9c5-fd12-4930-a2fa-23416ad90c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:41.419236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c0d9c5-fd12-4930-a2fa-23416ad90c7d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:41.430794 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1349/2680] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:22:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:22:41.435039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf4732-1001-473d-8240-c221f226a6f3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:41.445421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cf4732-1001-473d-8240-c221f226a6f3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:41.450131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0348c-3b44-40b4-a021-e8a751431987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:41.450717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0348c-3b44-40b4-a021-e8a751431987 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:41.450992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0348c-3b44-40b4-a021-e8a751431987 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:41.451261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0348c-3b44-40b4-a021-e8a751431987 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:41.451898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0348c-3b44-40b4-a021-e8a751431987 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:41.454380 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0348c-3b44-40b4-a021-e8a751431987 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:41.481676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1332/2681] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:22:41 2022] POST /identity/v3/projects => generated 359 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:22:41.485952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0d9c5-fd12-4930-a2fa-23416ad90c7d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:41.495866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c0d9c5-fd12-4930-a2fa-23416ad90c7d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:41.552537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7f9e4-fa3f-4eaf-b606-5ce27513e34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:41.553167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7f9e4-fa3f-4eaf-b606-5ce27513e34d admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:41.553447 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7f9e4-fa3f-4eaf-b606-5ce27513e34d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:41.553717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c7f9e4-fa3f-4eaf-b606-5ce27513e34d admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:41.554506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7f9e4-fa3f-4eaf-b606-5ce27513e34d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:41.556408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c7f9e4-fa3f-4eaf-b606-5ce27513e34d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:41.578951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1350/2682] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:22:41 2022] POST /identity/v3/users => generated 280 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:41.583302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0348c-3b44-40b4-a021-e8a751431987 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:41.593475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0348c-3b44-40b4-a021-e8a751431987 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:41.598166 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3a57c-1914-4481-9e00-628e7c0ecfbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:41.598757 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3a57c-1914-4481-9e00-628e7c0ecfbe admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:41.599033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3a57c-1914-4481-9e00-628e7c0ecfbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:41.599301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb3a57c-1914-4481-9e00-628e7c0ecfbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:41.599936 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3a57c-1914-4481-9e00-628e7c0ecfbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:41.601559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb3a57c-1914-4481-9e00-628e7c0ecfbe admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:41.611608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1333/2683] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:41.615500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7f9e4-fa3f-4eaf-b606-5ce27513e34d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:41.625796 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c7f9e4-fa3f-4eaf-b606-5ce27513e34d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:41.630408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b9007a-3d49-4bc1-ac87-1e4d81ba4990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:41.630989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9007a-3d49-4bc1-ac87-1e4d81ba4990 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:41.631265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9007a-3d49-4bc1-ac87-1e4d81ba4990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:41.631535 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9007a-3d49-4bc1-ac87-1e4d81ba4990 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:41.632164 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9007a-3d49-4bc1-ac87-1e4d81ba4990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:41.633771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b9007a-3d49-4bc1-ac87-1e4d81ba4990 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:41.644615 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1351/2684] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:22:41.648530 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3a57c-1914-4481-9e00-628e7c0ecfbe admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:41.658566 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb3a57c-1914-4481-9e00-628e7c0ecfbe admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:41.663402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bca370-a79a-4b1b-ad74-b568b6552044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:41.664075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bca370-a79a-4b1b-ad74-b568b6552044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:41.664355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bca370-a79a-4b1b-ad74-b568b6552044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:41.664630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bca370-a79a-4b1b-ad74-b568b6552044 admin admin] PATH_INFO: `/v3/projects/77f4d8ed616b4141ba0459d0ec15d83c/users/e2df1f5608cf433ea323cd1407288148/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:41.680326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bca370-a79a-4b1b-ad74-b568b6552044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f4d8ed616b4141ba0459d0ec15d83c, user_id=e2df1f5608cf433ea323cd1407288148, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:41.682985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bca370-a79a-4b1b-ad74-b568b6552044 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:41.696023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1334/2685] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:41 2022] PUT /identity/v3/projects/77f4d8ed616b4141ba0459d0ec15d83c/users/e2df1f5608cf433ea323cd1407288148/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:22:41.706693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79f3df-f5b2-46d6-808e-6d66ac304f0b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:41.706986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79f3df-f5b2-46d6-808e-6d66ac304f0b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:41.707279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79f3df-f5b2-46d6-808e-6d66ac304f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:41.733937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-3e79f3df-f5b2-46d6-808e-6d66ac304f0b None None] MFA Rules not processed for user `e2df1f5608cf433ea323cd1407288148`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:41.735659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-3e79f3df-f5b2-46d6-808e-6d66ac304f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:41.798502 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:41.808996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1352/2686] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:22:41.843919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bca370-a79a-4b1b-ad74-b568b6552044 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:41.845745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e4bca370-a79a-4b1b-ad74-b568b6552044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:41.902194 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bca370-a79a-4b1b-ad74-b568b6552044 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:41.905687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4539d0c-6227-4a24-87e4-5a49d3515ad0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:41.906284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4539d0c-6227-4a24-87e4-5a49d3515ad0 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:41.906694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4539d0c-6227-4a24-87e4-5a49d3515ad0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:41.906966 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4539d0c-6227-4a24-87e4-5a49d3515ad0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:41.912021 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:41.916669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4539d0c-6227-4a24-87e4-5a49d3515ad0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:41.918225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4539d0c-6227-4a24-87e4-5a49d3515ad0 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:41.928792 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1335/2687] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:41 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:41.944523 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79f3df-f5b2-46d6-808e-6d66ac304f0b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:42.006387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79f3df-f5b2-46d6-808e-6d66ac304f0b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:42.010988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:42.011619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:42.011895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:42.012170 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] PATH_INFO: `/v3/users/e2df1f5608cf433ea323cd1407288148` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:42.013328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2df1f5608cf433ea323cd1407288148)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:42.015244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:42.064033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2df1f5608cf433ea323cd1407288148'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:42.064815 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2df1f5608cf433ea323cd1407288148'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:42.065508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2df1f5608cf433ea323cd1407288148'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:42.078485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2df1f5608cf433ea323cd1407288148'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:42.086343 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2df1f5608cf433ea323cd1407288148'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:42.105274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1353/2688] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:41 2022] DELETE /identity/v3/users/e2df1f5608cf433ea323cd1407288148 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:42.107203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03ea8d-74fa-4046-bcca-0e604622fe64 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:42.107587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03ea8d-74fa-4046-bcca-0e604622fe64 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:42.107958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d03ea8d-74fa-4046-bcca-0e604622fe64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:42.129865 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-3d03ea8d-74fa-4046-bcca-0e604622fe64 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:42.132615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3d03ea8d-74fa-4046-bcca-0e604622fe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:42.203738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1336/2689] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:42.214684 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:42.216102 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:42.274719 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd34ba40-7c84-4001-b337-97298b0b5761 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:42.279054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82326e-395f-4681-a4bb-26b422814476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:42.279731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82326e-395f-4681-a4bb-26b422814476 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:42.280006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82326e-395f-4681-a4bb-26b422814476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:42.280276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82326e-395f-4681-a4bb-26b422814476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:42.290271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82326e-395f-4681-a4bb-26b422814476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:42.291697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82326e-395f-4681-a4bb-26b422814476 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:42.302032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1354/2690] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:42.694493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03ea8d-74fa-4046-bcca-0e604622fe64 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:42.696091 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3d03ea8d-74fa-4046-bcca-0e604622fe64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:42.711542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d03ea8d-74fa-4046-bcca-0e604622fe64 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:42.716375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:42.717057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:42.717356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:42.717647 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] PATH_INFO: `/v3/projects/77f4d8ed616b4141ba0459d0ec15d83c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:42.719067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=77f4d8ed616b4141ba0459d0ec15d83c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:42.721001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:42.771171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77f4d8ed616b4141ba0459d0ec15d83c', 'request_id': 'req-2309a2fc-9ea0-43e2-b875-539a587ccc3f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:42.778570 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77f4d8ed616b4141ba0459d0ec15d83c', 'request_id': 'req-2309a2fc-9ea0-43e2-b875-539a587ccc3f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:42.785168 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77f4d8ed616b4141ba0459d0ec15d83c', 'request_id': 'req-2309a2fc-9ea0-43e2-b875-539a587ccc3f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:42.795680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] The token cache is being invalidate because project 77f4d8ed616b4141ba0459d0ec15d83c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:42.795965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:42.796602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2309a2fc-9ea0-43e2-b875-539a587ccc3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:42.798854 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1337/2691] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:42 2022] DELETE /identity/v3/projects/77f4d8ed616b4141ba0459d0ec15d83c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:42.803933 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35654d-7cd9-4a2e-bd68-852c453b47c7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:42.804417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35654d-7cd9-4a2e-bd68-852c453b47c7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:42.804572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35654d-7cd9-4a2e-bd68-852c453b47c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:42.825289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-dc35654d-7cd9-4a2e-bd68-852c453b47c7 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:42.826480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-dc35654d-7cd9-4a2e-bd68-852c453b47c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:42.838816 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:42.893190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1355/2692] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:42.902199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e082ac-85e0-4e7c-a593-f17bd9a083cb None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:42.902485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e082ac-85e0-4e7c-a593-f17bd9a083cb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:42.902761 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e082ac-85e0-4e7c-a593-f17bd9a083cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:42.922747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-12e082ac-85e0-4e7c-a593-f17bd9a083cb None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:42.923958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-12e082ac-85e0-4e7c-a593-f17bd9a083cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:42.931872 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1338/2693] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:42.936073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35654d-7cd9-4a2e-bd68-852c453b47c7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:42.946259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35654d-7cd9-4a2e-bd68-852c453b47c7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:42.950747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca434d4b-8c1d-4f3f-b54a-0ef4fdaa9aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:42.951337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca434d4b-8c1d-4f3f-b54a-0ef4fdaa9aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:42.951608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca434d4b-8c1d-4f3f-b54a-0ef4fdaa9aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:42.951874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca434d4b-8c1d-4f3f-b54a-0ef4fdaa9aa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:42.952511 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca434d4b-8c1d-4f3f-b54a-0ef4fdaa9aa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:22:42.952813 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca434d4b-8c1d-4f3f-b54a-0ef4fdaa9aa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:42.954888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca434d4b-8c1d-4f3f-b54a-0ef4fdaa9aa4 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:42.966208 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1356/2694] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:22:42.970839 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e082ac-85e0-4e7c-a593-f17bd9a083cb None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:42.980321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e082ac-85e0-4e7c-a593-f17bd9a083cb None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:42.987350 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ce1ef-56ba-437a-b165-f9a38ade60c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:42.987976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ce1ef-56ba-437a-b165-f9a38ade60c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:42.988317 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ce1ef-56ba-437a-b165-f9a38ade60c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:42.988641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ce1ef-56ba-437a-b165-f9a38ade60c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:42.989323 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ce1ef-56ba-437a-b165-f9a38ade60c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:42.991231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ce1ef-56ba-437a-b165-f9a38ade60c8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:43.023592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1339/2695] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:22:42 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:43.027881 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca434d4b-8c1d-4f3f-b54a-0ef4fdaa9aa4 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:43.039107 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca434d4b-8c1d-4f3f-b54a-0ef4fdaa9aa4 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:43.101790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8298e-c386-4057-81ab-ae684687ca4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:43.102558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8298e-c386-4057-81ab-ae684687ca4f admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.102841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8298e-c386-4057-81ab-ae684687ca4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.103112 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b8298e-c386-4057-81ab-ae684687ca4f admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.103755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8298e-c386-4057-81ab-ae684687ca4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:43.105688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b8298e-c386-4057-81ab-ae684687ca4f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:43.131777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1357/2696] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:22:43 2022] POST /identity/v3/users => generated 291 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:43.135544 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ce1ef-56ba-437a-b165-f9a38ade60c8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:43.148363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ce1ef-56ba-437a-b165-f9a38ade60c8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:43.153226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7e5ad-a2e7-4de6-9994-e7156e6df1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:43.153860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7e5ad-a2e7-4de6-9994-e7156e6df1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.154139 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7e5ad-a2e7-4de6-9994-e7156e6df1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.154413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7e5ad-a2e7-4de6-9994-e7156e6df1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.155055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7e5ad-a2e7-4de6-9994-e7156e6df1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:43.157391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7e5ad-a2e7-4de6-9994-e7156e6df1b8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:43.169990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1340/2697] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:22:43.175461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8298e-c386-4057-81ab-ae684687ca4f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:43.187315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b8298e-c386-4057-81ab-ae684687ca4f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:43.192331 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a632afe0-7072-4cee-94c8-ffe62d9ddafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:43.192945 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632afe0-7072-4cee-94c8-ffe62d9ddafc admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.193258 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632afe0-7072-4cee-94c8-ffe62d9ddafc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.193531 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a632afe0-7072-4cee-94c8-ffe62d9ddafc admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.194178 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632afe0-7072-4cee-94c8-ffe62d9ddafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:43.195766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a632afe0-7072-4cee-94c8-ffe62d9ddafc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:43.210538 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1358/2698] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:43.212020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7e5ad-a2e7-4de6-9994-e7156e6df1b8 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:43.226240 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7e5ad-a2e7-4de6-9994-e7156e6df1b8 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:43.231410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99df291-1635-4e56-bb3a-a10dbad7f01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:43.232060 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99df291-1635-4e56-bb3a-a10dbad7f01c admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.232339 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99df291-1635-4e56-bb3a-a10dbad7f01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.232606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99df291-1635-4e56-bb3a-a10dbad7f01c admin admin] PATH_INFO: `/v3/projects/683de3acf3114dd1aed5f8e2fe49919d/users/40d61b2e1c984ceea9ad591647106b6c/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.258127 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99df291-1635-4e56-bb3a-a10dbad7f01c admin admin] RBAC: Authorizing `identity:create_grant(project_id=683de3acf3114dd1aed5f8e2fe49919d, user_id=40d61b2e1c984ceea9ad591647106b6c, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:43.261183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99df291-1635-4e56-bb3a-a10dbad7f01c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:43.275597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1341/2699] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:43 2022] PUT /identity/v3/projects/683de3acf3114dd1aed5f8e2fe49919d/users/40d61b2e1c984ceea9ad591647106b6c/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:43.286068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0741c5f-62ea-48df-a943-d720687070c6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.286353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0741c5f-62ea-48df-a943-d720687070c6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.286632 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0741c5f-62ea-48df-a943-d720687070c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.316127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b0741c5f-62ea-48df-a943-d720687070c6 None None] MFA Rules not processed for user `40d61b2e1c984ceea9ad591647106b6c`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:43.317448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b0741c5f-62ea-48df-a943-d720687070c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:43.389630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1359/2700] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:43.395346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99df291-1635-4e56-bb3a-a10dbad7f01c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:43.409089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:43.462068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99df291-1635-4e56-bb3a-a10dbad7f01c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:43.466778 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979047f5-cc0d-40e9-abc2-d43bff57c120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:43.467379 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979047f5-cc0d-40e9-abc2-d43bff57c120 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.467654 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979047f5-cc0d-40e9-abc2-d43bff57c120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.467923 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979047f5-cc0d-40e9-abc2-d43bff57c120 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.468593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979047f5-cc0d-40e9-abc2-d43bff57c120 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:43.471188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979047f5-cc0d-40e9-abc2-d43bff57c120 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:43.503985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1342/2701] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:22:43 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:43.505172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0741c5f-62ea-48df-a943-d720687070c6 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:43.516690 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0741c5f-62ea-48df-a943-d720687070c6 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:43.526159 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:43.576668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55110762-0a50-4a16-ac89-899b9f14a42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:43.577410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55110762-0a50-4a16-ac89-899b9f14a42e admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.577689 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55110762-0a50-4a16-ac89-899b9f14a42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.577960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55110762-0a50-4a16-ac89-899b9f14a42e admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.579118 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55110762-0a50-4a16-ac89-899b9f14a42e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:43.581001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55110762-0a50-4a16-ac89-899b9f14a42e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:43.604220 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1360/2702] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:22:43 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:22:43.608375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979047f5-cc0d-40e9-abc2-d43bff57c120 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:43.619444 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979047f5-cc0d-40e9-abc2-d43bff57c120 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:43.624052 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75438a4f-5a89-4aec-957d-278559125ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:43.624649 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75438a4f-5a89-4aec-957d-278559125ad3 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.624958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75438a4f-5a89-4aec-957d-278559125ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.625405 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75438a4f-5a89-4aec-957d-278559125ad3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.626149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75438a4f-5a89-4aec-957d-278559125ad3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:43.627779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75438a4f-5a89-4aec-957d-278559125ad3 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:43.644156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1343/2703] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:43.648787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55110762-0a50-4a16-ac89-899b9f14a42e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:43.662312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55110762-0a50-4a16-ac89-899b9f14a42e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:43.667529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f707479-b0c2-4411-ab10-644a66fc9952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:43.668132 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707479-b0c2-4411-ab10-644a66fc9952 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.668412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707479-b0c2-4411-ab10-644a66fc9952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.668672 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f707479-b0c2-4411-ab10-644a66fc9952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.669360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f707479-b0c2-4411-ab10-644a66fc9952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:43.670938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f707479-b0c2-4411-ab10-644a66fc9952 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:43.686047 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1361/2704] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:43.687323 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75438a4f-5a89-4aec-957d-278559125ad3 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:43.700660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75438a4f-5a89-4aec-957d-278559125ad3 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:43.706648 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e61bd2-fa9a-402d-8e21-3f16d7423bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:43.707396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e61bd2-fa9a-402d-8e21-3f16d7423bba admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.707759 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e61bd2-fa9a-402d-8e21-3f16d7423bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.708266 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e61bd2-fa9a-402d-8e21-3f16d7423bba admin admin] PATH_INFO: `/v3/projects/2a30772281c642c39d89855ab5a824b7/users/7a45b8e1fb8c4523b936b572f8d2ad08/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.726385 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e61bd2-fa9a-402d-8e21-3f16d7423bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a30772281c642c39d89855ab5a824b7, user_id=7a45b8e1fb8c4523b936b572f8d2ad08, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:43.728905 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e61bd2-fa9a-402d-8e21-3f16d7423bba admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:43.742284 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1344/2705] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:43 2022] PUT /identity/v3/projects/2a30772281c642c39d89855ab5a824b7/users/7a45b8e1fb8c4523b936b572f8d2ad08/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:43.753963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd012b-ee88-4fc7-9e8e-6fe6e4fb65c2 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.754251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd012b-ee88-4fc7-9e8e-6fe6e4fb65c2 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.754549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bd012b-ee88-4fc7-9e8e-6fe6e4fb65c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.785854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-62bd012b-ee88-4fc7-9e8e-6fe6e4fb65c2 None None] MFA Rules not processed for user `7a45b8e1fb8c4523b936b572f8d2ad08`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:43.787218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-62bd012b-ee88-4fc7-9e8e-6fe6e4fb65c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:43.856172 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1362/2706] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:43.860943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e61bd2-fa9a-402d-8e21-3f16d7423bba admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:43.936141 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e61bd2-fa9a-402d-8e21-3f16d7423bba admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:43.941786 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa69a2-b6b5-4e8d-9209-dbb511a138f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:43.942472 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa69a2-b6b5-4e8d-9209-dbb511a138f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:43.942818 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa69a2-b6b5-4e8d-9209-dbb511a138f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:43.943156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fa69a2-b6b5-4e8d-9209-dbb511a138f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:43.943858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa69a2-b6b5-4e8d-9209-dbb511a138f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:43.945927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fa69a2-b6b5-4e8d-9209-dbb511a138f3 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:43.981627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1345/2707] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:22:43 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:43.982840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd012b-ee88-4fc7-9e8e-6fe6e4fb65c2 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:43.997952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bd012b-ee88-4fc7-9e8e-6fe6e4fb65c2 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:44.089249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c85322-8e28-4e72-b725-5c4c07c6fe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:44.089977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c85322-8e28-4e72-b725-5c4c07c6fe30 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:44.090322 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c85322-8e28-4e72-b725-5c4c07c6fe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:44.090674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c85322-8e28-4e72-b725-5c4c07c6fe30 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:44.091393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c85322-8e28-4e72-b725-5c4c07c6fe30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:44.093457 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c85322-8e28-4e72-b725-5c4c07c6fe30 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:44.120262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1363/2708] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:22:43 2022] POST /identity/v3/users => generated 299 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:22:44.124177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa69a2-b6b5-4e8d-9209-dbb511a138f3 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:44.136723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fa69a2-b6b5-4e8d-9209-dbb511a138f3 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:44.142320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b829af5-64fb-4ed6-9d15-8f6d994054b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:44.142942 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b829af5-64fb-4ed6-9d15-8f6d994054b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:44.143823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b829af5-64fb-4ed6-9d15-8f6d994054b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:44.143823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b829af5-64fb-4ed6-9d15-8f6d994054b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:44.144126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b829af5-64fb-4ed6-9d15-8f6d994054b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:44.145781 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b829af5-64fb-4ed6-9d15-8f6d994054b0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:44.158520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1346/2709] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:44.162712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c85322-8e28-4e72-b725-5c4c07c6fe30 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:44.175016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c85322-8e28-4e72-b725-5c4c07c6fe30 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:44.181002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedca2d-e9f6-452f-b74d-af07c35b0b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:44.181002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedca2d-e9f6-452f-b74d-af07c35b0b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:44.181347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedca2d-e9f6-452f-b74d-af07c35b0b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:44.181630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedca2d-e9f6-452f-b74d-af07c35b0b3b admin admin] PATH_INFO: `/v3/projects/8897693887d74d8ab5f3793623aaa850/users/027665c333894d85b45edb4c6ad20a03/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:44.191887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:44.200320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bedca2d-e9f6-452f-b74d-af07c35b0b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8897693887d74d8ab5f3793623aaa850, user_id=027665c333894d85b45edb4c6ad20a03, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:44.203370 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bedca2d-e9f6-452f-b74d-af07c35b0b3b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:44.217032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1364/2710] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:44 2022] PUT /identity/v3/projects/8897693887d74d8ab5f3793623aaa850/users/027665c333894d85b45edb4c6ad20a03/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:44.220881 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b829af5-64fb-4ed6-9d15-8f6d994054b0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:44.231204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b829af5-64fb-4ed6-9d15-8f6d994054b0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:44.265308 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:44.289375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4e6199-3255-4583-b04b-2f94a634e80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:44.289985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e6199-3255-4583-b04b-2f94a634e80b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:44.290262 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e6199-3255-4583-b04b-2f94a634e80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:44.290533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4e6199-3255-4583-b04b-2f94a634e80b admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:44.291185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e6199-3255-4583-b04b-2f94a634e80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:44.292766 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4e6199-3255-4583-b04b-2f94a634e80b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:44.303765 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1347/2711] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:22:44.308192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedca2d-e9f6-452f-b74d-af07c35b0b3b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:44.319256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedca2d-e9f6-452f-b74d-af07c35b0b3b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:44.323938 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f3110-49a9-4f32-9509-d92aba2d3b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:44.324577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f3110-49a9-4f32-9509-d92aba2d3b8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:44.324850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f3110-49a9-4f32-9509-d92aba2d3b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:44.325129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3f3110-49a9-4f32-9509-d92aba2d3b8e admin admin] PATH_INFO: `/v3/projects/8897693887d74d8ab5f3793623aaa850/users/027665c333894d85b45edb4c6ad20a03/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:44.327033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f3110-49a9-4f32-9509-d92aba2d3b8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8897693887d74d8ab5f3793623aaa850, user_id=027665c333894d85b45edb4c6ad20a03, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:44.329710 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3f3110-49a9-4f32-9509-d92aba2d3b8e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:44.340245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1365/2712] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:22:44 2022] PUT /identity/v3/projects/8897693887d74d8ab5f3793623aaa850/users/027665c333894d85b45edb4c6ad20a03/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:22:44.350827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f533f9b-8cf0-4571-b7a7-3929811e0a8e None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:44.351117 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f533f9b-8cf0-4571-b7a7-3929811e0a8e None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:44.351389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f533f9b-8cf0-4571-b7a7-3929811e0a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:44.380550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9f533f9b-8cf0-4571-b7a7-3929811e0a8e None None] MFA Rules not processed for user `027665c333894d85b45edb4c6ad20a03`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:44.381784 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9f533f9b-8cf0-4571-b7a7-3929811e0a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:44.450902 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1348/2713] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:44.463166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f3110-49a9-4f32-9509-d92aba2d3b8e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:44.464907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1b3f3110-49a9-4f32-9509-d92aba2d3b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:44.529507 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3f3110-49a9-4f32-9509-d92aba2d3b8e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:44.534421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ae7b-35f3-41c6-905a-dee161eaef24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:44.535029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ae7b-35f3-41c6-905a-dee161eaef24 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:44.535302 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ae7b-35f3-41c6-905a-dee161eaef24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:44.535578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f473ae7b-35f3-41c6-905a-dee161eaef24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:44.597895 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473ae7b-35f3-41c6-905a-dee161eaef24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:44.599348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f473ae7b-35f3-41c6-905a-dee161eaef24 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:44.608966 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1366/2714] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:46.164889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f533f9b-8cf0-4571-b7a7-3929811e0a8e None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:46.167116 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9f533f9b-8cf0-4571-b7a7-3929811e0a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:46.188813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f533f9b-8cf0-4571-b7a7-3929811e0a8e None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:46.194906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1668ea-ef6f-42ad-8068-20e63b0858b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:46.195724 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1668ea-ef6f-42ad-8068-20e63b0858b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:46.196322 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1668ea-ef6f-42ad-8068-20e63b0858b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:46.197391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1668ea-ef6f-42ad-8068-20e63b0858b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:46.289910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1668ea-ef6f-42ad-8068-20e63b0858b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:46.291432 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1668ea-ef6f-42ad-8068-20e63b0858b9 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:46.302366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1349/2715] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:46.858458 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ae7b-35f3-41c6-905a-dee161eaef24 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:46.870021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f473ae7b-35f3-41c6-905a-dee161eaef24 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:46.875712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a572ff-706e-421a-92cd-ae2c242ffcff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:46.877960 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a572ff-706e-421a-92cd-ae2c242ffcff service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:46.878235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a572ff-706e-421a-92cd-ae2c242ffcff service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:46.878501 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a572ff-706e-421a-92cd-ae2c242ffcff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:46.894367 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a572ff-706e-421a-92cd-ae2c242ffcff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:46.896011 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a572ff-706e-421a-92cd-ae2c242ffcff service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:46.905988 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1367/2716] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:48.940486 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84480138-604e-4844-b88a-eb2c50a0dc8c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:48.941028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84480138-604e-4844-b88a-eb2c50a0dc8c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:48.941028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84480138-604e-4844-b88a-eb2c50a0dc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:48.960686 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-84480138-604e-4844-b88a-eb2c50a0dc8c None None] MFA Rules not processed for user `1b01f21ffa814a5da3873e5387938b2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:48.961919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-84480138-604e-4844-b88a-eb2c50a0dc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:49.016088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1350/2717] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:49.321268 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a572ff-706e-421a-92cd-ae2c242ffcff service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:49.322822 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-02a572ff-706e-421a-92cd-ae2c242ffcff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:49.380354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a572ff-706e-421a-92cd-ae2c242ffcff service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:49.383958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aac021-8138-44a1-8567-daa15ca1af7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:49.384557 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aac021-8138-44a1-8567-daa15ca1af7e service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:49.384841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aac021-8138-44a1-8567-daa15ca1af7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:49.385186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aac021-8138-44a1-8567-daa15ca1af7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:49.386818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-82aac021-8138-44a1-8567-daa15ca1af7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:49.397273 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aac021-8138-44a1-8567-daa15ca1af7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:49.398699 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aac021-8138-44a1-8567-daa15ca1af7e service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:49.406812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1368/2718] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:50.960455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84480138-604e-4844-b88a-eb2c50a0dc8c None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:50.964652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=100 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:51.034279 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84480138-604e-4844-b88a-eb2c50a0dc8c None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:51.039045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:51.039671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:51.039953 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:51.040226 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] PATH_INFO: `/v3/users/40d61b2e1c984ceea9ad591647106b6c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:51.041388 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=40d61b2e1c984ceea9ad591647106b6c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:51.043383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:51.095198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40d61b2e1c984ceea9ad591647106b6c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.096614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40d61b2e1c984ceea9ad591647106b6c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.103660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40d61b2e1c984ceea9ad591647106b6c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.105550 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40d61b2e1c984ceea9ad591647106b6c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.119802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40d61b2e1c984ceea9ad591647106b6c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.134093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1351/2719] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:50 2022] DELETE /identity/v3/users/40d61b2e1c984ceea9ad591647106b6c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:51.138705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273c89e-a17a-4dbf-857c-25f2f534c370 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:51.139176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273c89e-a17a-4dbf-857c-25f2f534c370 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:51.139296 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6273c89e-a17a-4dbf-857c-25f2f534c370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:51.159308 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6273c89e-a17a-4dbf-857c-25f2f534c370 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:51.160520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6273c89e-a17a-4dbf-857c-25f2f534c370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:51.172548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:51.226396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1369/2720] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:22:51.236736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:51.238171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:51.297624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1714d71-b025-448f-aac2-1617d94bba4b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:51.301605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7934bb-1d4b-4b3c-8d1b-09f84d8d95eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:51.302196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7934bb-1d4b-4b3c-8d1b-09f84d8d95eb service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:51.302468 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7934bb-1d4b-4b3c-8d1b-09f84d8d95eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:51.302740 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7934bb-1d4b-4b3c-8d1b-09f84d8d95eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:51.311827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7934bb-1d4b-4b3c-8d1b-09f84d8d95eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:51.313306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7934bb-1d4b-4b3c-8d1b-09f84d8d95eb service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:51.323811 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1352/2721] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:22:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:22:51.643744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273c89e-a17a-4dbf-857c-25f2f534c370 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:51.645323 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6273c89e-a17a-4dbf-857c-25f2f534c370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:51.662417 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6273c89e-a17a-4dbf-857c-25f2f534c370 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:51.668951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:51.669709 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:51.669986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:51.670254 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] PATH_INFO: `/v3/projects/683de3acf3114dd1aed5f8e2fe49919d` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:51.671496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] RBAC: Authorizing `identity:delete_project(project_id=683de3acf3114dd1aed5f8e2fe49919d)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:51.674235 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:51.724859 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '683de3acf3114dd1aed5f8e2fe49919d', 'request_id': 'req-9b08dc57-1090-40d3-9106-a0adf60fe639'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.731051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '683de3acf3114dd1aed5f8e2fe49919d', 'request_id': 'req-9b08dc57-1090-40d3-9106-a0adf60fe639'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.739135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '683de3acf3114dd1aed5f8e2fe49919d', 'request_id': 'req-9b08dc57-1090-40d3-9106-a0adf60fe639'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.754670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] The token cache is being invalidate because project 683de3acf3114dd1aed5f8e2fe49919d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:51.754961 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.755595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.757066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1370/2722] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:51 2022] DELETE /identity/v3/projects/683de3acf3114dd1aed5f8e2fe49919d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:51.761025 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7934bb-1d4b-4b3c-8d1b-09f84d8d95eb service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:51.770878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7934bb-1d4b-4b3c-8d1b-09f84d8d95eb service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:51.792138 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:51.834978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:51.835631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:51.835973 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:51.836307 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] PATH_INFO: `/v3/users/7a45b8e1fb8c4523b936b572f8d2ad08` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:51.837617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a45b8e1fb8c4523b936b572f8d2ad08)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:51.839897 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:51.903804 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a45b8e1fb8c4523b936b572f8d2ad08'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.904496 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a45b8e1fb8c4523b936b572f8d2ad08'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.918803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a45b8e1fb8c4523b936b572f8d2ad08'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.920008 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a45b8e1fb8c4523b936b572f8d2ad08'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.938110 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a45b8e1fb8c4523b936b572f8d2ad08'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:51.959451 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1353/2723] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:51 2022] DELETE /identity/v3/users/7a45b8e1fb8c4523b936b572f8d2ad08 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:52.276819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:52.278562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:52.300625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b08dc57-1090-40d3-9106-a0adf60fe639 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:52.368351 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:52.369231 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:52.369792 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:52.370238 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] PATH_INFO: `/v3/projects/2a30772281c642c39d89855ab5a824b7` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:52.371761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a30772281c642c39d89855ab5a824b7)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:52.373838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:52.421467 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:52.426340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a30772281c642c39d89855ab5a824b7', 'request_id': 'req-86af28aa-2d0d-40c6-a647-7fc3be51a67f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:52.435306 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a30772281c642c39d89855ab5a824b7', 'request_id': 'req-86af28aa-2d0d-40c6-a647-7fc3be51a67f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:52.442328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a30772281c642c39d89855ab5a824b7', 'request_id': 'req-86af28aa-2d0d-40c6-a647-7fc3be51a67f'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:52.457437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] The token cache is being invalidate because project 2a30772281c642c39d89855ab5a824b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:52.457731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:52.458415 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:52.460366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1371/2724] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:52 2022] DELETE /identity/v3/projects/2a30772281c642c39d89855ab5a824b7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:52.464363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:52.467123 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:52.483033 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096288ed-8d26-487a-ae54-0e1885f5809a admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:52.544100 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9702905b-5ec6-4c09-98d2-ffd0e78bb250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:52.544736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702905b-5ec6-4c09-98d2-ffd0e78bb250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:52.545017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702905b-5ec6-4c09-98d2-ffd0e78bb250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:52.545343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9702905b-5ec6-4c09-98d2-ffd0e78bb250 admin admin] PATH_INFO: `/v3/users/027665c333894d85b45edb4c6ad20a03` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:52.546559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702905b-5ec6-4c09-98d2-ffd0e78bb250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=027665c333894d85b45edb4c6ad20a03)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:52.548504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9702905b-5ec6-4c09-98d2-ffd0e78bb250 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:52.600005 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:22:52.601790 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9702905b-5ec6-4c09-98d2-ffd0e78bb250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '027665c333894d85b45edb4c6ad20a03'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:52.602646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9702905b-5ec6-4c09-98d2-ffd0e78bb250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '027665c333894d85b45edb4c6ad20a03'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:52.612065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9702905b-5ec6-4c09-98d2-ffd0e78bb250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '027665c333894d85b45edb4c6ad20a03'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:52.612728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9702905b-5ec6-4c09-98d2-ffd0e78bb250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '027665c333894d85b45edb4c6ad20a03'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:52.626384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9702905b-5ec6-4c09-98d2-ffd0e78bb250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '027665c333894d85b45edb4c6ad20a03'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:52.640253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1354/2725] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:22:52 2022] DELETE /identity/v3/users/027665c333894d85b45edb4c6ad20a03 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:52.941066 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:22:52.942703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:52.956628 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86af28aa-2d0d-40c6-a647-7fc3be51a67f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:22:53.015171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:22:53.015809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:53.016090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:53.016364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] PATH_INFO: `/v3/projects/8897693887d74d8ab5f3793623aaa850` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:53.017662 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=8897693887d74d8ab5f3793623aaa850)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:22:53.019748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:22:53.067870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8897693887d74d8ab5f3793623aaa850', 'request_id': 'req-1952fc19-04b1-494e-91cb-266a27a772ca'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:53.074095 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8897693887d74d8ab5f3793623aaa850', 'request_id': 'req-1952fc19-04b1-494e-91cb-266a27a772ca'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:53.082073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8897693887d74d8ab5f3793623aaa850', 'request_id': 'req-1952fc19-04b1-494e-91cb-266a27a772ca'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:53.095021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] The token cache is being invalidate because project 8897693887d74d8ab5f3793623aaa850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:22:53.095311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:53.096422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1952fc19-04b1-494e-91cb-266a27a772ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:22:53.098004 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1372/2726] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:22:52 2022] DELETE /identity/v3/projects/8897693887d74d8ab5f3793623aaa850 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:22:58.269756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a77fe-bc24-4568-bb44-146802cb97c0 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:22:58.270632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a77fe-bc24-4568-bb44-146802cb97c0 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:22:58.270949 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a77fe-bc24-4568-bb44-146802cb97c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:22:58.292487 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-705a77fe-bc24-4568-bb44-146802cb97c0 None None] MFA Rules not processed for user `59d6a1d6b9bc4daf8542b58cf62fff1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:22:58.293828 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-705a77fe-bc24-4568-bb44-146802cb97c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:22:58.350202 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1355/2727] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:22:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:23:03.091826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=59,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:08.342889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=38,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:24.068288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47977cf-6f5b-49da-b7d7-d6e8691fd0f6 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:24.068783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47977cf-6f5b-49da-b7d7-d6e8691fd0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:24.068926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47977cf-6f5b-49da-b7d7-d6e8691fd0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:24.089490 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b47977cf-6f5b-49da-b7d7-d6e8691fd0f6 None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:23:24.090701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b47977cf-6f5b-49da-b7d7-d6e8691fd0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:24.145776 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1373/2728] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:23:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:23:24.194666 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a77fe-bc24-4568-bb44-146802cb97c0 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:24.196133 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-705a77fe-bc24-4568-bb44-146802cb97c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:24.268098 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a77fe-bc24-4568-bb44-146802cb97c0 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:24.273516 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2cfc2-7aba-4a78-ab03-ef437951ebeb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:24.274681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2cfc2-7aba-4a78-ab03-ef437951ebeb service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:24.274958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2cfc2-7aba-4a78-ab03-ef437951ebeb service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:24.275230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d2cfc2-7aba-4a78-ab03-ef437951ebeb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:24.284285 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2cfc2-7aba-4a78-ab03-ef437951ebeb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:24.285942 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d2cfc2-7aba-4a78-ab03-ef437951ebeb service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:24.295420 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1356/2729] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:23:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:23:34.139136 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=25 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:34.292055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=32 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:39.826473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2d733-6ad0-455f-afd0-b728caeb5415 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:39.826959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2d733-6ad0-455f-afd0-b728caeb5415 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:39.827069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2d733-6ad0-455f-afd0-b728caeb5415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:39.849334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-abd2d733-6ad0-455f-afd0-b728caeb5415 None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:23:39.850539 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-abd2d733-6ad0-455f-afd0-b728caeb5415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:39.858365 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1374/2730] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 22 22:23:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:23:39.870789 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2cfc2-7aba-4a78-ab03-ef437951ebeb service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:39.881237 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d2cfc2-7aba-4a78-ab03-ef437951ebeb service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:39.886616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fd329-c896-4590-98d4-a2d63c88546e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:39.887203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fd329-c896-4590-98d4-a2d63c88546e service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:39.887484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fd329-c896-4590-98d4-a2d63c88546e service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:39.887754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fd329-c896-4590-98d4-a2d63c88546e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:39.895671 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fd329-c896-4590-98d4-a2d63c88546e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:39.897277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fd329-c896-4590-98d4-a2d63c88546e service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:39.905978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1357/2731] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:23:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:23:41.753674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2d733-6ad0-455f-afd0-b728caeb5415 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:41.755105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-abd2d733-6ad0-455f-afd0-b728caeb5415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:41.815418 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fd329-c896-4590-98d4-a2d63c88546e service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:41.818273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d73fd329-c896-4590-98d4-a2d63c88546e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:41.829396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2d733-6ad0-455f-afd0-b728caeb5415 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:41.835275 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da357ef-9c33-4d4c-a517-51bcfc3b5ab0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:41.835890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da357ef-9c33-4d4c-a517-51bcfc3b5ab0 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:41.836166 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da357ef-9c33-4d4c-a517-51bcfc3b5ab0 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:41.836434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da357ef-9c33-4d4c-a517-51bcfc3b5ab0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:41.839175 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2da357ef-9c33-4d4c-a517-51bcfc3b5ab0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:41.851520 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da357ef-9c33-4d4c-a517-51bcfc3b5ab0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:41.853567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da357ef-9c33-4d4c-a517-51bcfc3b5ab0 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:41.863782 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1375/2732] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:23:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:23:41.886404 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fd329-c896-4590-98d4-a2d63c88546e service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:41.891066 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a73c1-98fc-458a-aef3-0e8036aa7d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:41.891729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a73c1-98fc-458a-aef3-0e8036aa7d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:41.891998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a73c1-98fc-458a-aef3-0e8036aa7d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:41.892274 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a73c1-98fc-458a-aef3-0e8036aa7d99 admin admin] PATH_INFO: `/v3/users/59d6a1d6b9bc4daf8542b58cf62fff1a` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:41.893436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a73c1-98fc-458a-aef3-0e8036aa7d99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59d6a1d6b9bc4daf8542b58cf62fff1a)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:41.895431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a73c1-98fc-458a-aef3-0e8036aa7d99 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:41.942997 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-568a73c1-98fc-458a-aef3-0e8036aa7d99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59d6a1d6b9bc4daf8542b58cf62fff1a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:41.943650 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-568a73c1-98fc-458a-aef3-0e8036aa7d99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59d6a1d6b9bc4daf8542b58cf62fff1a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:41.951545 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-568a73c1-98fc-458a-aef3-0e8036aa7d99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59d6a1d6b9bc4daf8542b58cf62fff1a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:41.952167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-568a73c1-98fc-458a-aef3-0e8036aa7d99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59d6a1d6b9bc4daf8542b58cf62fff1a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:41.964969 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-568a73c1-98fc-458a-aef3-0e8036aa7d99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59d6a1d6b9bc4daf8542b58cf62fff1a'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:41.977517 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1358/2733] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:23:41 2022] DELETE /identity/v3/users/59d6a1d6b9bc4daf8542b58cf62fff1a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:23:42.148325 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da357ef-9c33-4d4c-a517-51bcfc3b5ab0 service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:42.160890 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da357ef-9c33-4d4c-a517-51bcfc3b5ab0 service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:42.222622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:42.223252 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:42.223532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:42.223804 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] PATH_INFO: `/v3/projects/ed0761f1f2a148ce8b55f519a13421c8` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:42.224915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed0761f1f2a148ce8b55f519a13421c8)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:42.226840 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:42.272943 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed0761f1f2a148ce8b55f519a13421c8', 'request_id': 'req-1c48a261-ff36-4a08-831e-6d81288b14cf'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:42.279005 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed0761f1f2a148ce8b55f519a13421c8', 'request_id': 'req-1c48a261-ff36-4a08-831e-6d81288b14cf'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:42.284506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed0761f1f2a148ce8b55f519a13421c8', 'request_id': 'req-1c48a261-ff36-4a08-831e-6d81288b14cf'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:42.296759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] The token cache is being invalidate because project ed0761f1f2a148ce8b55f519a13421c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:23:42.297048 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:42.297755 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-1c48a261-ff36-4a08-831e-6d81288b14cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:42.299205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1376/2734] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:23:42 2022] DELETE /identity/v3/projects/ed0761f1f2a148ce8b55f519a13421c8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:23:42.304149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73654314-8362-4cf6-8166-6cd291a0f1fe None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:42.304445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73654314-8362-4cf6-8166-6cd291a0f1fe None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:42.304723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73654314-8362-4cf6-8166-6cd291a0f1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:42.324967 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-73654314-8362-4cf6-8166-6cd291a0f1fe None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:23:42.326250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-73654314-8362-4cf6-8166-6cd291a0f1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:42.384562 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1359/2735] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:23:42.392060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ce71a-6739-4fd3-83f5-2965cbd0cc60 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:42.392288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ce71a-6739-4fd3-83f5-2965cbd0cc60 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:42.392558 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71ce71a-6739-4fd3-83f5-2965cbd0cc60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:42.411882 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-b71ce71a-6739-4fd3-83f5-2965cbd0cc60 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:23:42.413070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b71ce71a-6739-4fd3-83f5-2965cbd0cc60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:42.421393 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1377/2736] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:23:42.425413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73654314-8362-4cf6-8166-6cd291a0f1fe None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:42.435587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73654314-8362-4cf6-8166-6cd291a0f1fe None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:42.440231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0edbf9-fe2c-424e-aece-7c26c71442c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:42.440815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0edbf9-fe2c-424e-aece-7c26c71442c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:42.441094 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0edbf9-fe2c-424e-aece-7c26c71442c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:42.441394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0edbf9-fe2c-424e-aece-7c26c71442c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:42.442028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0edbf9-fe2c-424e-aece-7c26c71442c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:23:42.442328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0edbf9-fe2c-424e-aece-7c26c71442c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:42.443925 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0edbf9-fe2c-424e-aece-7c26c71442c4 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:42.445627 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:42.460477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1360/2737] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:23:42.462022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ce71a-6739-4fd3-83f5-2965cbd0cc60 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:42.471850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71ce71a-6739-4fd3-83f5-2965cbd0cc60 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:42.476597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f94b6-54d4-4fc2-aa23-ec19fdf91e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:42.477209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f94b6-54d4-4fc2-aa23-ec19fdf91e84 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:42.477489 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f94b6-54d4-4fc2-aa23-ec19fdf91e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:42.477764 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f94b6-54d4-4fc2-aa23-ec19fdf91e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:42.479123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f94b6-54d4-4fc2-aa23-ec19fdf91e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:42.481037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f94b6-54d4-4fc2-aa23-ec19fdf91e84 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:42.485560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:42.510909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1378/2738] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:23:42 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:23:42.515082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0edbf9-fe2c-424e-aece-7c26c71442c4 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:42.525024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0edbf9-fe2c-424e-aece-7c26c71442c4 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:42.583602 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22652e-7b7c-4b9b-869d-ec9c216a486f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:42.584196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22652e-7b7c-4b9b-869d-ec9c216a486f admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:42.584476 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22652e-7b7c-4b9b-869d-ec9c216a486f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:42.584749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e22652e-7b7c-4b9b-869d-ec9c216a486f admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:42.585424 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22652e-7b7c-4b9b-869d-ec9c216a486f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:42.587318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e22652e-7b7c-4b9b-869d-ec9c216a486f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:42.608071 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1361/2739] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:23:42 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:23:42.612133 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f94b6-54d4-4fc2-aa23-ec19fdf91e84 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:42.622963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f94b6-54d4-4fc2-aa23-ec19fdf91e84 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:42.627488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2658c-a480-4ba7-b8d1-d17535ad30ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:42.628074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2658c-a480-4ba7-b8d1-d17535ad30ff admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:42.628348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2658c-a480-4ba7-b8d1-d17535ad30ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:42.628616 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf2658c-a480-4ba7-b8d1-d17535ad30ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:42.629301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2658c-a480-4ba7-b8d1-d17535ad30ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:42.630863 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf2658c-a480-4ba7-b8d1-d17535ad30ff admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:42.640536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1379/2740] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:23:42.644389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22652e-7b7c-4b9b-869d-ec9c216a486f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:42.654863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e22652e-7b7c-4b9b-869d-ec9c216a486f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:42.659460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f72696-0938-4341-b55d-cc0a749315ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:42.660039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f72696-0938-4341-b55d-cc0a749315ac admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:42.660318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f72696-0938-4341-b55d-cc0a749315ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:42.660608 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f72696-0938-4341-b55d-cc0a749315ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:42.661269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f72696-0938-4341-b55d-cc0a749315ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:42.662849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f72696-0938-4341-b55d-cc0a749315ac admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:42.673927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1362/2741] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:23:42.677822 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2658c-a480-4ba7-b8d1-d17535ad30ff admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:42.687651 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf2658c-a480-4ba7-b8d1-d17535ad30ff admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:42.692217 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76fdf60-df3b-46d5-a854-7b128bbf0e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:42.692845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fdf60-df3b-46d5-a854-7b128bbf0e37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:42.693145 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fdf60-df3b-46d5-a854-7b128bbf0e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:42.693418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76fdf60-df3b-46d5-a854-7b128bbf0e37 admin admin] PATH_INFO: `/v3/projects/c11f0dca10d74f8d91e534d5a065a12f/users/3d402ba519934c37a1c8158faabae529/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:42.707717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76fdf60-df3b-46d5-a854-7b128bbf0e37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c11f0dca10d74f8d91e534d5a065a12f, user_id=3d402ba519934c37a1c8158faabae529, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:42.710350 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76fdf60-df3b-46d5-a854-7b128bbf0e37 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:42.723000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1380/2742] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:23:42 2022] PUT /identity/v3/projects/c11f0dca10d74f8d91e534d5a065a12f/users/3d402ba519934c37a1c8158faabae529/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:23:42.734538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858d2a9-d1ad-46be-9510-450bc3b801b9 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:42.734822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858d2a9-d1ad-46be-9510-450bc3b801b9 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:42.735095 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e858d2a9-d1ad-46be-9510-450bc3b801b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:42.764486 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e858d2a9-d1ad-46be-9510-450bc3b801b9 None None] MFA Rules not processed for user `3d402ba519934c37a1c8158faabae529`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:23:42.765728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e858d2a9-d1ad-46be-9510-450bc3b801b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:42.828952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1363/2743] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:23:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:23:42.839887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76fdf60-df3b-46d5-a854-7b128bbf0e37 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:42.841637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b76fdf60-df3b-46d5-a854-7b128bbf0e37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:42.897770 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76fdf60-df3b-46d5-a854-7b128bbf0e37 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:42.901485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83d65a4-b4e8-4daf-8e09-ab3ff43c6890 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:42.902081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d65a4-b4e8-4daf-8e09-ab3ff43c6890 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:42.902354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d65a4-b4e8-4daf-8e09-ab3ff43c6890 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:42.902622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d65a4-b4e8-4daf-8e09-ab3ff43c6890 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:42.910705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d65a4-b4e8-4daf-8e09-ab3ff43c6890 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:42.912097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83d65a4-b4e8-4daf-8e09-ab3ff43c6890 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:42.920784 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1381/2744] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:23:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:23:45.470930 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858d2a9-d1ad-46be-9510-450bc3b801b9 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:45.531833 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e858d2a9-d1ad-46be-9510-450bc3b801b9 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:45.536320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:45.536935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:45.537246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:45.537518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] PATH_INFO: `/v3/users/3d402ba519934c37a1c8158faabae529` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:45.538630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d402ba519934c37a1c8158faabae529)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:45.540516 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:45.543501 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:45.588414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d402ba519934c37a1c8158faabae529'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:45.589057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d402ba519934c37a1c8158faabae529'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:45.596106 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d402ba519934c37a1c8158faabae529'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:45.596749 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d402ba519934c37a1c8158faabae529'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:45.609391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d402ba519934c37a1c8158faabae529'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:45.623454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1364/2745] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:23:45 2022] DELETE /identity/v3/users/3d402ba519934c37a1c8158faabae529 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:23:45.627932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb0de-f92c-4e60-b057-f3229f1b5dd5 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:45.628416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb0de-f92c-4e60-b057-f3229f1b5dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:45.628527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110eb0de-f92c-4e60-b057-f3229f1b5dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:45.647167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-110eb0de-f92c-4e60-b057-f3229f1b5dd5 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:23:45.648363 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-110eb0de-f92c-4e60-b057-f3229f1b5dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:45.706266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1382/2746] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:23:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:23:45.716474 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:45.717887 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:45.773629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcedabf-39fa-44d8-b838-94734ad6a1ed admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:45.778029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a0460-8f49-4ac8-a0ed-99b95c0de82b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:45.778634 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a0460-8f49-4ac8-a0ed-99b95c0de82b service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:45.778910 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a0460-8f49-4ac8-a0ed-99b95c0de82b service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:45.779180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a0460-8f49-4ac8-a0ed-99b95c0de82b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:45.788056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a0460-8f49-4ac8-a0ed-99b95c0de82b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:45.789508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a0460-8f49-4ac8-a0ed-99b95c0de82b service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:45.799691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1365/2747] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:23:46.071301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110eb0de-f92c-4e60-b057-f3229f1b5dd5 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:46.073436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-110eb0de-f92c-4e60-b057-f3229f1b5dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:46.085065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110eb0de-f92c-4e60-b057-f3229f1b5dd5 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:46.090285 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:46.090908 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:46.091179 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:46.091450 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] PATH_INFO: `/v3/projects/c11f0dca10d74f8d91e534d5a065a12f` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:46.092567 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=c11f0dca10d74f8d91e534d5a065a12f)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:46.094481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:46.143008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:46.144387 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c11f0dca10d74f8d91e534d5a065a12f', 'request_id': 'req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:46.153260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c11f0dca10d74f8d91e534d5a065a12f', 'request_id': 'req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:46.159314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c11f0dca10d74f8d91e534d5a065a12f', 'request_id': 'req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:46.171717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] The token cache is being invalidate because project c11f0dca10d74f8d91e534d5a065a12f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:23:46.171993 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:46.172623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-595d4e51-7a50-48fa-b06f-aaeaf5f878ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:46.174138 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1383/2748] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:23:46 2022] DELETE /identity/v3/projects/c11f0dca10d74f8d91e534d5a065a12f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:23:46.179308 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc68386-bd22-4c82-91e8-55f6f8d25050 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:46.179593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc68386-bd22-4c82-91e8-55f6f8d25050 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:46.179868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc68386-bd22-4c82-91e8-55f6f8d25050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:46.199706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-8bc68386-bd22-4c82-91e8-55f6f8d25050 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:23:46.200901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8bc68386-bd22-4c82-91e8-55f6f8d25050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:46.258310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1366/2749] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:23:46.266265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abd760-44fc-44fb-8134-00ad3c407097 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:46.266555 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abd760-44fc-44fb-8134-00ad3c407097 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:46.266833 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06abd760-44fc-44fb-8134-00ad3c407097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:46.287180 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-06abd760-44fc-44fb-8134-00ad3c407097 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:23:46.288390 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-06abd760-44fc-44fb-8134-00ad3c407097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:46.296364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1384/2750] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:23:46.300633 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc68386-bd22-4c82-91e8-55f6f8d25050 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:46.310209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc68386-bd22-4c82-91e8-55f6f8d25050 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:46.314691 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b964118-2b67-4389-a962-dc7e23a6f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:46.315268 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b964118-2b67-4389-a962-dc7e23a6f34b admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:46.315542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b964118-2b67-4389-a962-dc7e23a6f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:46.315806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b964118-2b67-4389-a962-dc7e23a6f34b admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:46.316432 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b964118-2b67-4389-a962-dc7e23a6f34b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:23:46.316727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b964118-2b67-4389-a962-dc7e23a6f34b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:46.318341 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b964118-2b67-4389-a962-dc7e23a6f34b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:46.330587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1367/2751] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:23:46.334688 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abd760-44fc-44fb-8134-00ad3c407097 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:46.344805 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06abd760-44fc-44fb-8134-00ad3c407097 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:46.350103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99374b-cae3-4a15-9bab-689b15d1d356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:46.350727 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99374b-cae3-4a15-9bab-689b15d1d356 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:46.351016 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99374b-cae3-4a15-9bab-689b15d1d356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:46.351318 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99374b-cae3-4a15-9bab-689b15d1d356 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:46.352169 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99374b-cae3-4a15-9bab-689b15d1d356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:46.354320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99374b-cae3-4a15-9bab-689b15d1d356 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:46.381830 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1385/2752] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:23:46 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:23:46.386636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b964118-2b67-4389-a962-dc7e23a6f34b admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:46.396482 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b964118-2b67-4389-a962-dc7e23a6f34b admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:46.404303 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:46.454504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e7c41-792f-4ace-b458-b56ef46723aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:46.455100 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e7c41-792f-4ace-b458-b56ef46723aa admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:46.455373 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e7c41-792f-4ace-b458-b56ef46723aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:46.455641 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566e7c41-792f-4ace-b458-b56ef46723aa admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:46.456279 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e7c41-792f-4ace-b458-b56ef46723aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:46.458204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566e7c41-792f-4ace-b458-b56ef46723aa admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:46.479264 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1368/2753] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:23:46 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:23:46.483376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99374b-cae3-4a15-9bab-689b15d1d356 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:46.493633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99374b-cae3-4a15-9bab-689b15d1d356 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:46.498983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82593449-c2fb-4357-ba9d-65c50b8dd15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:46.499576 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82593449-c2fb-4357-ba9d-65c50b8dd15a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:46.499849 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82593449-c2fb-4357-ba9d-65c50b8dd15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:46.500118 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82593449-c2fb-4357-ba9d-65c50b8dd15a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:46.500758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82593449-c2fb-4357-ba9d-65c50b8dd15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:46.502388 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82593449-c2fb-4357-ba9d-65c50b8dd15a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:46.512977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1386/2754] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:23:46.516684 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e7c41-792f-4ace-b458-b56ef46723aa admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:46.526754 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566e7c41-792f-4ace-b458-b56ef46723aa admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:46.531592 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa2de28-b90a-47bf-bdf3-80adfd0e3f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:46.532178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2de28-b90a-47bf-bdf3-80adfd0e3f48 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:46.532456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2de28-b90a-47bf-bdf3-80adfd0e3f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:46.532730 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa2de28-b90a-47bf-bdf3-80adfd0e3f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:46.533394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2de28-b90a-47bf-bdf3-80adfd0e3f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:46.534995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa2de28-b90a-47bf-bdf3-80adfd0e3f48 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:46.545882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1369/2755] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:23:46.549980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82593449-c2fb-4357-ba9d-65c50b8dd15a admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:46.560707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82593449-c2fb-4357-ba9d-65c50b8dd15a admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:46.565218 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a5da-781d-4e8a-83a3-4dbf6356b127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:46.565854 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a5da-781d-4e8a-83a3-4dbf6356b127 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:46.566130 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a5da-781d-4e8a-83a3-4dbf6356b127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:46.566396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a5da-781d-4e8a-83a3-4dbf6356b127 admin admin] PATH_INFO: `/v3/projects/48cd248992874994a490170d08d0c791/users/5a7236271e6c474f80728e9dc970d67b/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:46.581488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345a5da-781d-4e8a-83a3-4dbf6356b127 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cd248992874994a490170d08d0c791, user_id=5a7236271e6c474f80728e9dc970d67b, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:46.584078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345a5da-781d-4e8a-83a3-4dbf6356b127 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:46.596730 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1387/2756] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:23:46 2022] PUT /identity/v3/projects/48cd248992874994a490170d08d0c791/users/5a7236271e6c474f80728e9dc970d67b/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:23:46.606625 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e8deb-1942-4ae3-a0c5-60f3d93b7c57 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:46.606909 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e8deb-1942-4ae3-a0c5-60f3d93b7c57 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:46.607183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e8deb-1942-4ae3-a0c5-60f3d93b7c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:46.634375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-a85e8deb-1942-4ae3-a0c5-60f3d93b7c57 None None] MFA Rules not processed for user `5a7236271e6c474f80728e9dc970d67b`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:23:46.636502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a85e8deb-1942-4ae3-a0c5-60f3d93b7c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:46.698937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1370/2757] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:23:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:23:46.709581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a5da-781d-4e8a-83a3-4dbf6356b127 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:46.711236 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c345a5da-781d-4e8a-83a3-4dbf6356b127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:46.766571 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a5da-781d-4e8a-83a3-4dbf6356b127 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:46.770156 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0ab6b6-59ba-42ab-a32c-427279842e37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:46.770745 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ab6b6-59ba-42ab-a32c-427279842e37 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:46.771017 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ab6b6-59ba-42ab-a32c-427279842e37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:46.771282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0ab6b6-59ba-42ab-a32c-427279842e37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:46.779671 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ab6b6-59ba-42ab-a32c-427279842e37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:46.781074 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0ab6b6-59ba-42ab-a32c-427279842e37 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:46.789758 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1388/2758] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:23:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:23:56.695242 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:56.785777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=60,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:56.846802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e8deb-1942-4ae3-a0c5-60f3d93b7c57 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:56.912739 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e8deb-1942-4ae3-a0c5-60f3d93b7c57 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:56.918000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:56.918638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:56.918916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:56.919184 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] PATH_INFO: `/v3/users/5a7236271e6c474f80728e9dc970d67b` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:56.920345 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a7236271e6c474f80728e9dc970d67b)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:56.922448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:56.974505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a7236271e6c474f80728e9dc970d67b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:56.975225 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a7236271e6c474f80728e9dc970d67b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:56.984104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a7236271e6c474f80728e9dc970d67b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:56.984875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a7236271e6c474f80728e9dc970d67b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:56.998089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a7236271e6c474f80728e9dc970d67b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:57.011408 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1371/2759] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:23:56 2022] DELETE /identity/v3/users/5a7236271e6c474f80728e9dc970d67b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:23:57.016039 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a3b51-2f8a-40af-968a-9b708e7e0973 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:57.016320 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a3b51-2f8a-40af-968a-9b708e7e0973 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:57.016589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a3b51-2f8a-40af-968a-9b708e7e0973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:57.038584 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a91a3b51-2f8a-40af-968a-9b708e7e0973 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:23:57.040122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a91a3b51-2f8a-40af-968a-9b708e7e0973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:57.108007 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1389/2760] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:23:57.120250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:57.121856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:57.186200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1822b754-ee8d-4497-958a-95896ecc6aaa admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:57.189774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f725e-9646-40c4-9467-a8d8c0c842b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:57.190366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f725e-9646-40c4-9467-a8d8c0c842b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:57.190637 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f725e-9646-40c4-9467-a8d8c0c842b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:57.190901 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f725e-9646-40c4-9467-a8d8c0c842b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:57.200607 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f725e-9646-40c4-9467-a8d8c0c842b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:57.202057 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f725e-9646-40c4-9467-a8d8c0c842b1 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:57.211167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1372/2761] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:23:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:23:57.566015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a3b51-2f8a-40af-968a-9b708e7e0973 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:57.567998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a91a3b51-2f8a-40af-968a-9b708e7e0973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:57.582082 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a3b51-2f8a-40af-968a-9b708e7e0973 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:57.586902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:57.587537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:57.587819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:57.588115 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] PATH_INFO: `/v3/projects/48cd248992874994a490170d08d0c791` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:57.589738 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=48cd248992874994a490170d08d0c791)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:57.591665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:57.639069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48cd248992874994a490170d08d0c791', 'request_id': 'req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:57.645072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48cd248992874994a490170d08d0c791', 'request_id': 'req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:57.650627 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48cd248992874994a490170d08d0c791', 'request_id': 'req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:57.663629 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] The token cache is being invalidate because project 48cd248992874994a490170d08d0c791 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:23:57.663917 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:57.664541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:23:57.666051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1390/2762] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:23:57 2022] DELETE /identity/v3/projects/48cd248992874994a490170d08d0c791 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:23:58.977041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f725e-9646-40c4-9467-a8d8c0c842b1 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:23:58.978521 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-a40f725e-9646-40c4-9467-a8d8c0c842b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:59.004467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:23:59.050271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f725e-9646-40c4-9467-a8d8c0c842b1 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:23:59.108939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d824f-f4f2-479e-9953-4245cf9e9a70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:23:59.109567 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d824f-f4f2-479e-9953-4245cf9e9a70 service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:23:59.109844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d824f-f4f2-479e-9953-4245cf9e9a70 service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:23:59.110114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d824f-f4f2-479e-9953-4245cf9e9a70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:23:59.111845 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6a7d824f-f4f2-479e-9953-4245cf9e9a70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:23:59.123629 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d824f-f4f2-479e-9953-4245cf9e9a70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:23:59.125221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d824f-f4f2-479e-9953-4245cf9e9a70 service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:23:59.134965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1373/2763] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:23:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:24:07.660647 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:24:08.317399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:24:08.318920 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:08.383900 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b26ee8-3005-4e0a-b2ed-2f1365a16a2d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:24:08.388563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed8cf0-bb87-40c1-bf02-5b04220f837f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:24:08.389209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed8cf0-bb87-40c1-bf02-5b04220f837f service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:08.389492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed8cf0-bb87-40c1-bf02-5b04220f837f service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:08.389766 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ed8cf0-bb87-40c1-bf02-5b04220f837f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:08.391566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-26ed8cf0-bb87-40c1-bf02-5b04220f837f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:08.462663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed8cf0-bb87-40c1-bf02-5b04220f837f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:24:08.464115 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ed8cf0-bb87-40c1-bf02-5b04220f837f service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:24:08.474590 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1391/2764] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:24:09.130822 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=48 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:24:14.832122 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d824f-f4f2-479e-9953-4245cf9e9a70 service placement] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:24:14.833699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6a7d824f-f4f2-479e-9953-4245cf9e9a70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:14.848028 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d824f-f4f2-479e-9953-4245cf9e9a70 service placement] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:24:14.853906 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:24:14.854523 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:14.854799 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:14.855062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] PATH_INFO: `/v3/users/cc89baeb82444da78312c4b0b3fb0c41` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:14.856181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc89baeb82444da78312c4b0b3fb0c41)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:24:14.858218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:24:14.915183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc89baeb82444da78312c4b0b3fb0c41'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:14.915836 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc89baeb82444da78312c4b0b3fb0c41'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:14.922835 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc89baeb82444da78312c4b0b3fb0c41'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:14.923457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc89baeb82444da78312c4b0b3fb0c41'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:14.935727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc89baeb82444da78312c4b0b3fb0c41'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:14.948593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1374/2765] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:24:14 2022] DELETE /identity/v3/users/cc89baeb82444da78312c4b0b3fb0c41 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:24:15.127600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed8cf0-bb87-40c1-bf02-5b04220f837f service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:24:15.138953 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ed8cf0-bb87-40c1-bf02-5b04220f837f service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:24:15.196488 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:24:15.197122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:15.197427 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:15.197701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] PATH_INFO: `/v3/projects/e9c311fa04f241249d428df8e88539f4` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:15.199068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9c311fa04f241249d428df8e88539f4)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:24:15.201798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:24:15.246638 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9c311fa04f241249d428df8e88539f4', 'request_id': 'req-2e4e8b16-14ff-4101-ab68-875440c5ef7b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.249059 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:24:15.253851 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9c311fa04f241249d428df8e88539f4', 'request_id': 'req-2e4e8b16-14ff-4101-ab68-875440c5ef7b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.259984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9c311fa04f241249d428df8e88539f4', 'request_id': 'req-2e4e8b16-14ff-4101-ab68-875440c5ef7b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.272008 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] The token cache is being invalidate because project e9c311fa04f241249d428df8e88539f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:24:15.272297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.272912 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.274400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1392/2766] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:24:15 2022] DELETE /identity/v3/projects/e9c311fa04f241249d428df8e88539f4 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:24:15.278611 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:24:15.280045 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:15.292885 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c753ff6-6683-4d1b-9bcd-758d6ccf1e59 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:24:15.354448 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:24:15.354992 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:15.355265 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:15.355529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] PATH_INFO: `/v3/users/1c20d023d8fc4b31b7f306ee58e3bada` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:15.357326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c20d023d8fc4b31b7f306ee58e3bada)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:24:15.359240 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:24:15.404000 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c20d023d8fc4b31b7f306ee58e3bada'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.404639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c20d023d8fc4b31b7f306ee58e3bada'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.411453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c20d023d8fc4b31b7f306ee58e3bada'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.412087 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c20d023d8fc4b31b7f306ee58e3bada'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.424027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c20d023d8fc4b31b7f306ee58e3bada'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.431814 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:24:15.444320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1375/2767] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:24:15 2022] DELETE /identity/v3/users/1c20d023d8fc4b31b7f306ee58e3bada => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:24:15.615146 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:24:15.616705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:15.631238 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e8b16-14ff-4101-ab68-875440c5ef7b admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:24:15.687375 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:24:15.688001 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:15.688280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:15.688553 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] PATH_INFO: `/v3/projects/316ac566e6c247c899de1498fdd81206` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:15.689686 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] RBAC: Authorizing `identity:delete_project(project_id=316ac566e6c247c899de1498fdd81206)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:24:15.691566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:24:15.737219 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '316ac566e6c247c899de1498fdd81206', 'request_id': 'req-517a663a-4fb5-44c6-893b-18a6895d9432'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.743149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '316ac566e6c247c899de1498fdd81206', 'request_id': 'req-517a663a-4fb5-44c6-893b-18a6895d9432'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.748643 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '316ac566e6c247c899de1498fdd81206', 'request_id': 'req-517a663a-4fb5-44c6-893b-18a6895d9432'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.762382 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] The token cache is being invalidate because project 316ac566e6c247c899de1498fdd81206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:24:15.762671 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.763301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-517a663a-4fb5-44c6-893b-18a6895d9432 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:24:15.764756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1393/2768] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:24:15 2022] DELETE /identity/v3/projects/316ac566e6c247c899de1498fdd81206 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:24:20.453216 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:24:20.454907 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:20.528233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8be0d3-5d40-4316-b0b0-0504078cf94f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:24:20.589745 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1afeed-a755-4b26-b75c-c5bc7b33e0ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:24:20.590430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1afeed-a755-4b26-b75c-c5bc7b33e0ea service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:20.590708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1afeed-a755-4b26-b75c-c5bc7b33e0ea service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:20.590976 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1afeed-a755-4b26-b75c-c5bc7b33e0ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:20.593455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bd1afeed-a755-4b26-b75c-c5bc7b33e0ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:20.649690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1afeed-a755-4b26-b75c-c5bc7b33e0ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:24:20.651247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1afeed-a755-4b26-b75c-c5bc7b33e0ea service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:24:20.660883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1376/2769] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:24:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:24:25.759311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:24:30.656727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=86 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:24:39.140371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4476a-54cc-4b72-be4b-aa3f77514cfa None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:39.141391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4476a-54cc-4b72-be4b-aa3f77514cfa None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:39.141635 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f4476a-54cc-4b72-be4b-aa3f77514cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:39.162774 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-c7f4476a-54cc-4b72-be4b-aa3f77514cfa None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:24:39.164057 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c7f4476a-54cc-4b72-be4b-aa3f77514cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:39.218239 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1394/2770] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:24:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:24:39.240527 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1afeed-a755-4b26-b75c-c5bc7b33e0ea service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:24:39.241959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bd1afeed-a755-4b26-b75c-c5bc7b33e0ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:39.308808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1afeed-a755-4b26-b75c-c5bc7b33e0ea service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:24:39.314324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e00b01-ef10-40a4-a9c4-66e36469150a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:24:39.314913 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e00b01-ef10-40a4-a9c4-66e36469150a service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:39.315185 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e00b01-ef10-40a4-a9c4-66e36469150a service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:39.315453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e00b01-ef10-40a4-a9c4-66e36469150a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:39.324310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e00b01-ef10-40a4-a9c4-66e36469150a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:24:39.326180 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e00b01-ef10-40a4-a9c4-66e36469150a service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:24:39.336169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1377/2771] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:24:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:24:49.211736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=25 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:24:49.332610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=32 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:24:54.536566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0b5ad-7ca8-4a6e-8299-e86998b172a4 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:54.537193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0b5ad-7ca8-4a6e-8299-e86998b172a4 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:54.537193 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0b5ad-7ca8-4a6e-8299-e86998b172a4 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:54.538478 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1395/2772] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:24:54.544861 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c6804-84db-47f6-a2c3-668add77713c None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:54.545368 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c6804-84db-47f6-a2c3-668add77713c None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:54.545603 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995c6804-84db-47f6-a2c3-668add77713c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:54.568306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-995c6804-84db-47f6-a2c3-668add77713c None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:24:54.569672 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-995c6804-84db-47f6-a2c3-668add77713c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:54.579187 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1378/2773] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:24:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:24:54.590741 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0b5ad-7ca8-4a6e-8299-e86998b172a4 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:24:54.592562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-94f0b5ad-7ca8-4a6e-8299-e86998b172a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:54.652685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0b5ad-7ca8-4a6e-8299-e86998b172a4 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:24:54.656221 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19d0ed-a506-44b3-81b1-0f9fd3619be0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '195565a7695b4d9b82e0ee76181fee2f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:24:54.656820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d0ed-a506-44b3-81b1-0f9fd3619be0 service glance] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:54.657091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d0ed-a506-44b3-81b1-0f9fd3619be0 service glance] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:54.657380 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19d0ed-a506-44b3-81b1-0f9fd3619be0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:54.667644 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19d0ed-a506-44b3-81b1-0f9fd3619be0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:24:54.669053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c19d0ed-a506-44b3-81b1-0f9fd3619be0 service glance] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:24:54.680537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1396/2774] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:24:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:24:54.748086 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0223b-24f2-47a3-bcaa-82100349c060 None None] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:54.748331 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0223b-24f2-47a3-bcaa-82100349c060 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:54.748605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0223b-24f2-47a3-bcaa-82100349c060 None None] PATH_INFO: `/` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:54.749990 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1379/2775] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 22 22:24:54.755697 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2f2b8-66d6-4df3-9a73-07f44db50f1d None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:54.756072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2f2b8-66d6-4df3-9a73-07f44db50f1d None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:54.756311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2f2b8-66d6-4df3-9a73-07f44db50f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:54.775683 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-a4e2f2b8-66d6-4df3-9a73-07f44db50f1d None None] MFA Rules not processed for user `18c956fa035f40fa88fbf662541121ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:24:54.776892 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-a4e2f2b8-66d6-4df3-9a73-07f44db50f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:54.827666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1397/2776] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:24:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:24:54.839394 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0223b-24f2-47a3-bcaa-82100349c060 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:24:54.841043 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-3cf0223b-24f2-47a3-bcaa-82100349c060 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:54.854688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0223b-24f2-47a3-bcaa-82100349c060 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:24:54.858204 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7852c57-3368-41bc-a3a3-cf0f07075ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:24:54.858783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7852c57-3368-41bc-a3a3-cf0f07075ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:54.859055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7852c57-3368-41bc-a3a3-cf0f07075ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:54.859319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7852c57-3368-41bc-a3a3-cf0f07075ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:54.866714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7852c57-3368-41bc-a3a3-cf0f07075ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:24:54.868107 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7852c57-3368-41bc-a3a3-cf0f07075ea5 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:24:54.875728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1380/2777] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:24:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:24:54.957824 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c74c-591b-4550-a01c-1d9558e77229 None None] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:54.958151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c74c-591b-4550-a01c-1d9558e77229 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:54.958430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892c74c-591b-4550-a01c-1d9558e77229 None None] PATH_INFO: `/` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:54.959781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1398/2778] 10.222.0.25 () {54 vars in 921 bytes} [Thu Sep 22 22:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 22 22:24:54.964755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921b3f-4aa7-4525-ac8c-2e1419f3bf57 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:54.965041 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921b3f-4aa7-4525-ac8c-2e1419f3bf57 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:54.965354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd921b3f-4aa7-4525-ac8c-2e1419f3bf57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:54.985803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-bd921b3f-4aa7-4525-ac8c-2e1419f3bf57 None None] MFA Rules not processed for user `d011e54e12674eaeb82c68da628ecb9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:24:54.987989 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-bd921b3f-4aa7-4525-ac8c-2e1419f3bf57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:54.997324 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1381/2779] 10.222.0.25 () {58 vars in 1018 bytes} [Thu Sep 22 22:24:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:24:55.008018 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892c74c-591b-4550-a01c-1d9558e77229 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:24:55.019176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892c74c-591b-4550-a01c-1d9558e77229 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:24:55.023459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30193ba-463b-451e-bae7-77e8bb930f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:24:55.024058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30193ba-463b-451e-bae7-77e8bb930f8d service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:55.024336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30193ba-463b-451e-bae7-77e8bb930f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:55.029149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30193ba-463b-451e-bae7-77e8bb930f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:55.040740 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30193ba-463b-451e-bae7-77e8bb930f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:24:55.042222 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30193ba-463b-451e-bae7-77e8bb930f8d service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:24:55.053922 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1399/2780] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:24:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:24:56.768927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fedf9b-0e5f-4cfe-83e3-ab6642d2b7fb None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:56.769439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fedf9b-0e5f-4cfe-83e3-ab6642d2b7fb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:56.769439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fedf9b-0e5f-4cfe-83e3-ab6642d2b7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:56.788387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-43fedf9b-0e5f-4cfe-83e3-ab6642d2b7fb None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:24:56.789606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-43fedf9b-0e5f-4cfe-83e3-ab6642d2b7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:24:56.796542 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1382/2781] 10.222.0.25 () {58 vars in 1009 bytes} [Thu Sep 22 22:24:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:24:56.809579 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30193ba-463b-451e-bae7-77e8bb930f8d service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:24:56.820871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30193ba-463b-451e-bae7-77e8bb930f8d service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:24:56.826090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4929ea-6512-4aea-a01e-979b6cc224d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:24:56.826684 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4929ea-6512-4aea-a01e-979b6cc224d1 service swift] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:24:56.826959 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4929ea-6512-4aea-a01e-979b6cc224d1 service swift] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:24:56.827481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4929ea-6512-4aea-a01e-979b6cc224d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:24:56.836012 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4929ea-6512-4aea-a01e-979b6cc224d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:24:56.837594 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4929ea-6512-4aea-a01e-979b6cc224d1 service swift] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:24:56.845592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1400/2782] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:24:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 22 22:25:06.783107 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=15 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:25:06.842330 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=61 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:25:14.850306 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fedf9b-0e5f-4cfe-83e3-ab6642d2b7fb None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:25:14.852252 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-43fedf9b-0e5f-4cfe-83e3-ab6642d2b7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:25:14.863493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fedf9b-0e5f-4cfe-83e3-ab6642d2b7fb None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:25:14.867143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566cb51-fd41-425b-90d6-978813e74c71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:25:14.867916 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566cb51-fd41-425b-90d6-978813e74c71 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:25:14.868191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566cb51-fd41-425b-90d6-978813e74c71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:25:14.868458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f566cb51-fd41-425b-90d6-978813e74c71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:25:14.870452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-f566cb51-fd41-425b-90d6-978813e74c71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:25:14.884039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566cb51-fd41-425b-90d6-978813e74c71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:25:14.885493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f566cb51-fd41-425b-90d6-978813e74c71 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:25:14.898328 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1383/2783] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:25:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:25:14.908362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baab68a-9c84-41e6-ac94-ba8cacbb1346 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:25:14.912098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baab68a-9c84-41e6-ac94-ba8cacbb1346 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:25:14.912595 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baab68a-9c84-41e6-ac94-ba8cacbb1346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:25:14.933184 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-8baab68a-9c84-41e6-ac94-ba8cacbb1346 None None] MFA Rules not processed for user `bf3dde2bfc4240a8af6425778c5eddaf`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:25:14.934397 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-8baab68a-9c84-41e6-ac94-ba8cacbb1346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:25:14.941677 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1401/2784] 10.222.0.25 () {58 vars in 967 bytes} [Thu Sep 22 22:25:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:25:14.954272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566cb51-fd41-425b-90d6-978813e74c71 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:25:14.966106 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f566cb51-fd41-425b-90d6-978813e74c71 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:25:14.971609 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e2813-d51f-49c0-802f-e629303d5dca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '471e7550ae7a4f909851215d3ed148c6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:25:14.972334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e2813-d51f-49c0-802f-e629303d5dca service swift] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:25:14.972612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e2813-d51f-49c0-802f-e629303d5dca service swift] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:25:14.972880 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e2813-d51f-49c0-802f-e629303d5dca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:25:14.981964 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e2813-d51f-49c0-802f-e629303d5dca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:25:14.983536 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e2813-d51f-49c0-802f-e629303d5dca service swift] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:25:14.991891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1384/2785] 10.222.0.25 () {60 vars in 1384 bytes} [Thu Sep 22 22:25:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 22 22:25:16.088183 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baab68a-9c84-41e6-ac94-ba8cacbb1346 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:25:16.097715 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baab68a-9c84-41e6-ac94-ba8cacbb1346 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:25:16.101759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef877be-d9de-473c-a98f-bd5dfacaa2cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:25:16.102625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef877be-d9de-473c-a98f-bd5dfacaa2cc service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:25:16.103084 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef877be-d9de-473c-a98f-bd5dfacaa2cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:25:16.103521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef877be-d9de-473c-a98f-bd5dfacaa2cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:25:16.106091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7ef877be-d9de-473c-a98f-bd5dfacaa2cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:25:16.125759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef877be-d9de-473c-a98f-bd5dfacaa2cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:25:16.128010 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef877be-d9de-473c-a98f-bd5dfacaa2cc service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:25:16.142413 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1402/2786] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:25:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:25:24.988435 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=12 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:25:26.136834 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=9 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:18.349920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e2813-d51f-49c0-802f-e629303d5dca service swift] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:18.359150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e2813-d51f-49c0-802f-e629303d5dca service swift] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:18.362692 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f68eee-e651-4bfa-94a8-833dbb5b7b99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:18.363276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f68eee-e651-4bfa-94a8-833dbb5b7b99 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:18.367430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f68eee-e651-4bfa-94a8-833dbb5b7b99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:18.367430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f68eee-e651-4bfa-94a8-833dbb5b7b99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:18.367430 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-64f68eee-e651-4bfa-94a8-833dbb5b7b99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:18.430626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f68eee-e651-4bfa-94a8-833dbb5b7b99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:18.432059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f68eee-e651-4bfa-94a8-833dbb5b7b99 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:18.442171 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1385/2787] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:21.528207 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef877be-d9de-473c-a98f-bd5dfacaa2cc service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:21.530397 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7ef877be-d9de-473c-a98f-bd5dfacaa2cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:21.545032 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef877be-d9de-473c-a98f-bd5dfacaa2cc service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:21.549668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18550e96-dec8-4022-a0fa-b6afeea8ed75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:21.550301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18550e96-dec8-4022-a0fa-b6afeea8ed75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:21.550581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18550e96-dec8-4022-a0fa-b6afeea8ed75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:21.550856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18550e96-dec8-4022-a0fa-b6afeea8ed75 admin admin] PATH_INFO: `/v3/users/1b01f21ffa814a5da3873e5387938b2a` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:21.552002 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18550e96-dec8-4022-a0fa-b6afeea8ed75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b01f21ffa814a5da3873e5387938b2a)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:21.553972 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18550e96-dec8-4022-a0fa-b6afeea8ed75 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:21.603242 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-18550e96-dec8-4022-a0fa-b6afeea8ed75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b01f21ffa814a5da3873e5387938b2a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:21.603914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-18550e96-dec8-4022-a0fa-b6afeea8ed75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b01f21ffa814a5da3873e5387938b2a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:21.604515 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-18550e96-dec8-4022-a0fa-b6afeea8ed75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b01f21ffa814a5da3873e5387938b2a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:21.618602 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-18550e96-dec8-4022-a0fa-b6afeea8ed75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b01f21ffa814a5da3873e5387938b2a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:21.626373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-18550e96-dec8-4022-a0fa-b6afeea8ed75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b01f21ffa814a5da3873e5387938b2a'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:21.640636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1403/2788] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:21 2022] DELETE /identity/v3/users/1b01f21ffa814a5da3873e5387938b2a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:21.822577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f68eee-e651-4bfa-94a8-833dbb5b7b99 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:21.833896 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f68eee-e651-4bfa-94a8-833dbb5b7b99 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:21.894425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:21.895058 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:21.895340 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:21.895610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] PATH_INFO: `/v3/projects/eaff29c3c9b84929b0451e8e82caf4f2` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:21.896766 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaff29c3c9b84929b0451e8e82caf4f2)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:21.898720 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:21.946042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaff29c3c9b84929b0451e8e82caf4f2', 'request_id': 'req-5a60a89d-412e-45fe-856d-3e873723bb3b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:21.953343 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaff29c3c9b84929b0451e8e82caf4f2', 'request_id': 'req-5a60a89d-412e-45fe-856d-3e873723bb3b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:21.959700 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaff29c3c9b84929b0451e8e82caf4f2', 'request_id': 'req-5a60a89d-412e-45fe-856d-3e873723bb3b'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:21.969863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] The token cache is being invalidate because project eaff29c3c9b84929b0451e8e82caf4f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:21.970146 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:21.970770 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-5a60a89d-412e-45fe-856d-3e873723bb3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:21.972436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1386/2789] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:21 2022] DELETE /identity/v3/projects/eaff29c3c9b84929b0451e8e82caf4f2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:21.977437 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe5c04-b5f6-43a4-af5a-723635afb031 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:21.977722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe5c04-b5f6-43a4-af5a-723635afb031 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:21.977997 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe5c04-b5f6-43a4-af5a-723635afb031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:21.998421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-acbe5c04-b5f6-43a4-af5a-723635afb031 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:21.999634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-acbe5c04-b5f6-43a4-af5a-723635afb031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:22.069905 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1404/2790] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:26:22.078696 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c10c4-6991-4601-a898-1e14838d43b8 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.079065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c10c4-6991-4601-a898-1e14838d43b8 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.079416 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219c10c4-6991-4601-a898-1e14838d43b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.099985 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-219c10c4-6991-4601-a898-1e14838d43b8 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:22.101228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-219c10c4-6991-4601-a898-1e14838d43b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:22.109995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1387/2791] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:26:22.114566 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe5c04-b5f6-43a4-af5a-723635afb031 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.125048 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe5c04-b5f6-43a4-af5a-723635afb031 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.130617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7c578-883c-4ef1-9149-c35610a21335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.131251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7c578-883c-4ef1-9149-c35610a21335 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.131622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7c578-883c-4ef1-9149-c35610a21335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.131736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b7c578-883c-4ef1-9149-c35610a21335 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.132386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7c578-883c-4ef1-9149-c35610a21335 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:26:22.132687 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7c578-883c-4ef1-9149-c35610a21335 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.134313 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b7c578-883c-4ef1-9149-c35610a21335 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.145999 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1405/2792] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:26:22.150245 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c10c4-6991-4601-a898-1e14838d43b8 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.160135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219c10c4-6991-4601-a898-1e14838d43b8 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.164708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b593d-b435-4031-be8d-5948d4b6e451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.165335 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b593d-b435-4031-be8d-5948d4b6e451 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.165614 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b593d-b435-4031-be8d-5948d4b6e451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.165889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334b593d-b435-4031-be8d-5948d4b6e451 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.166514 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b593d-b435-4031-be8d-5948d4b6e451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.168399 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334b593d-b435-4031-be8d-5948d4b6e451 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.184391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:22.198580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1388/2793] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:22 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:22.202703 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7c578-883c-4ef1-9149-c35610a21335 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.212752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b7c578-883c-4ef1-9149-c35610a21335 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.268435 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e9cb6-2444-40a2-84ae-119f6e230da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.269029 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e9cb6-2444-40a2-84ae-119f6e230da0 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.269346 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e9cb6-2444-40a2-84ae-119f6e230da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.269619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e9cb6-2444-40a2-84ae-119f6e230da0 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.270266 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e9cb6-2444-40a2-84ae-119f6e230da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.272149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e9cb6-2444-40a2-84ae-119f6e230da0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.280589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:22.296874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1406/2794] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:22 2022] POST /identity/v3/users => generated 301 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:22.300944 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b593d-b435-4031-be8d-5948d4b6e451 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.311075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334b593d-b435-4031-be8d-5948d4b6e451 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.315883 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52609638-6369-4cbb-bd19-c3612a57f625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.316477 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609638-6369-4cbb-bd19-c3612a57f625 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.316755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609638-6369-4cbb-bd19-c3612a57f625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.317055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52609638-6369-4cbb-bd19-c3612a57f625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.317747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52609638-6369-4cbb-bd19-c3612a57f625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.319338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52609638-6369-4cbb-bd19-c3612a57f625 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.329248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1389/2795] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:22.333279 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e9cb6-2444-40a2-84ae-119f6e230da0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.343551 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e9cb6-2444-40a2-84ae-119f6e230da0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.348192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a71419-4c76-4103-9e1f-658fa32cfcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.348778 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a71419-4c76-4103-9e1f-658fa32cfcbc admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.349055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a71419-4c76-4103-9e1f-658fa32cfcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.349355 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a71419-4c76-4103-9e1f-658fa32cfcbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.349989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a71419-4c76-4103-9e1f-658fa32cfcbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.351550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a71419-4c76-4103-9e1f-658fa32cfcbc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.362767 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1407/2796] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:26:22.366996 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52609638-6369-4cbb-bd19-c3612a57f625 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.376931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52609638-6369-4cbb-bd19-c3612a57f625 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.381561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24c0ae-5ca2-4df3-bbd4-34dfd66035e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.382189 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24c0ae-5ca2-4df3-bbd4-34dfd66035e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.382464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24c0ae-5ca2-4df3-bbd4-34dfd66035e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.382733 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24c0ae-5ca2-4df3-bbd4-34dfd66035e3 admin admin] PATH_INFO: `/v3/projects/960cda3cd79c44dabf1748e44118aeb6/users/3bdd7befef39489a88a9703bc42fc1ce/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.397067 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24c0ae-5ca2-4df3-bbd4-34dfd66035e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=960cda3cd79c44dabf1748e44118aeb6, user_id=3bdd7befef39489a88a9703bc42fc1ce, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.399743 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24c0ae-5ca2-4df3-bbd4-34dfd66035e3 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.413346 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1390/2797] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:22 2022] PUT /identity/v3/projects/960cda3cd79c44dabf1748e44118aeb6/users/3bdd7befef39489a88a9703bc42fc1ce/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:22.424549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8358f-b7a2-4be6-84dc-748f2f12eff3 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.424829 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8358f-b7a2-4be6-84dc-748f2f12eff3 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.425098 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8358f-b7a2-4be6-84dc-748f2f12eff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.451910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-5ad8358f-b7a2-4be6-84dc-748f2f12eff3 None None] MFA Rules not processed for user `3bdd7befef39489a88a9703bc42fc1ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:22.453033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5ad8358f-b7a2-4be6-84dc-748f2f12eff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:22.516665 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1408/2798] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:22 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:22.521363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24c0ae-5ca2-4df3-bbd4-34dfd66035e3 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.589036 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24c0ae-5ca2-4df3-bbd4-34dfd66035e3 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.594451 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1982b-3976-4d46-a2ce-b616a80bec10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.595042 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1982b-3976-4d46-a2ce-b616a80bec10 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.595321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1982b-3976-4d46-a2ce-b616a80bec10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.595593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd1982b-3976-4d46-a2ce-b616a80bec10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.596230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1982b-3976-4d46-a2ce-b616a80bec10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.598143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd1982b-3976-4d46-a2ce-b616a80bec10 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.623816 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1391/2799] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:22 2022] POST /identity/v3/projects => generated 405 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:22.628090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8358f-b7a2-4be6-84dc-748f2f12eff3 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.640544 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8358f-b7a2-4be6-84dc-748f2f12eff3 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.698569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb0d98-ea27-46c2-94f4-62171ae2c78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.699167 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb0d98-ea27-46c2-94f4-62171ae2c78c admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.699441 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb0d98-ea27-46c2-94f4-62171ae2c78c admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.699707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb0d98-ea27-46c2-94f4-62171ae2c78c admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.700870 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb0d98-ea27-46c2-94f4-62171ae2c78c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.702845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcb0d98-ea27-46c2-94f4-62171ae2c78c admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.724404 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1409/2800] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:22 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:22.728319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1982b-3976-4d46-a2ce-b616a80bec10 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.738628 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd1982b-3976-4d46-a2ce-b616a80bec10 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.743351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29492a58-8731-486a-a456-d324fbbcb93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.743942 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29492a58-8731-486a-a456-d324fbbcb93f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.744220 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29492a58-8731-486a-a456-d324fbbcb93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.744490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29492a58-8731-486a-a456-d324fbbcb93f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.745145 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29492a58-8731-486a-a456-d324fbbcb93f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.746723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29492a58-8731-486a-a456-d324fbbcb93f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.757076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1392/2801] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:22.761037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb0d98-ea27-46c2-94f4-62171ae2c78c admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.771873 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb0d98-ea27-46c2-94f4-62171ae2c78c admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.776424 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda4342-b47d-45c2-884e-a0e6bb0d3651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.777060 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4342-b47d-45c2-884e-a0e6bb0d3651 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.777366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4342-b47d-45c2-884e-a0e6bb0d3651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.777634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda4342-b47d-45c2-884e-a0e6bb0d3651 admin admin] PATH_INFO: `/v3/projects/fecea78a61954c8e8caad253a02f98a3/users/f8efdb7f24fd452e868a161ddbc1839f/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.792439 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda4342-b47d-45c2-884e-a0e6bb0d3651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fecea78a61954c8e8caad253a02f98a3, user_id=f8efdb7f24fd452e868a161ddbc1839f, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.795151 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda4342-b47d-45c2-884e-a0e6bb0d3651 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.807549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1410/2802] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:22 2022] PUT /identity/v3/projects/fecea78a61954c8e8caad253a02f98a3/users/f8efdb7f24fd452e868a161ddbc1839f/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:22.811505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29492a58-8731-486a-a456-d324fbbcb93f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.821384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29492a58-8731-486a-a456-d324fbbcb93f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.880248 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f349e-6930-4da5-9976-cdff70c4f605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.880914 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f349e-6930-4da5-9976-cdff70c4f605 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.881131 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f349e-6930-4da5-9976-cdff70c4f605 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.881421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f349e-6930-4da5-9976-cdff70c4f605 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.882085 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f349e-6930-4da5-9976-cdff70c4f605 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.883725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953f349e-6930-4da5-9976-cdff70c4f605 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.894081 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1393/2803] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:22.898347 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda4342-b47d-45c2-884e-a0e6bb0d3651 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:22.908537 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda4342-b47d-45c2-884e-a0e6bb0d3651 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:22.915092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243a2c4-1886-4e4f-8ba7-b151ca9290c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:22.915747 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243a2c4-1886-4e4f-8ba7-b151ca9290c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.916022 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243a2c4-1886-4e4f-8ba7-b151ca9290c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.916292 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c243a2c4-1886-4e4f-8ba7-b151ca9290c6 admin admin] PATH_INFO: `/v3/projects/fecea78a61954c8e8caad253a02f98a3/users/f8efdb7f24fd452e868a161ddbc1839f/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.918245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243a2c4-1886-4e4f-8ba7-b151ca9290c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fecea78a61954c8e8caad253a02f98a3, user_id=f8efdb7f24fd452e868a161ddbc1839f, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:22.920852 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c243a2c4-1886-4e4f-8ba7-b151ca9290c6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:22.931120 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1411/2804] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:22 2022] PUT /identity/v3/projects/fecea78a61954c8e8caad253a02f98a3/users/f8efdb7f24fd452e868a161ddbc1839f/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:22.943078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31524c1-f60f-480e-8ee1-9cc666dfdcc5 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:22.943365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31524c1-f60f-480e-8ee1-9cc666dfdcc5 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:22.943631 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31524c1-f60f-480e-8ee1-9cc666dfdcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:22.971502 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-e31524c1-f60f-480e-8ee1-9cc666dfdcc5 None None] MFA Rules not processed for user `f8efdb7f24fd452e868a161ddbc1839f`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:22.972625 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-e31524c1-f60f-480e-8ee1-9cc666dfdcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:22.990326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:23.041581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1394/2805] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:22 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:23.052291 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243a2c4-1886-4e4f-8ba7-b151ca9290c6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:23.054062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-c243a2c4-1886-4e4f-8ba7-b151ca9290c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:23.077760 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:23.114971 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c243a2c4-1886-4e4f-8ba7-b151ca9290c6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:23.118582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6650a35c-143b-4fd9-b0ca-ce39a35ffd9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:23.119174 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a35c-143b-4fd9-b0ca-ce39a35ffd9e service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:23.119448 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a35c-143b-4fd9-b0ca-ce39a35ffd9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:23.119713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6650a35c-143b-4fd9-b0ca-ce39a35ffd9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:23.129128 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650a35c-143b-4fd9-b0ca-ce39a35ffd9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:23.130545 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6650a35c-143b-4fd9-b0ca-ce39a35ffd9e service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:23.140749 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1412/2806] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:23 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:23.288199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31524c1-f60f-480e-8ee1-9cc666dfdcc5 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:23.349148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31524c1-f60f-480e-8ee1-9cc666dfdcc5 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:23.353690 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:23.354304 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:23.354581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:23.354857 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] PATH_INFO: `/v3/users/3bdd7befef39489a88a9703bc42fc1ce` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:23.355972 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bdd7befef39489a88a9703bc42fc1ce)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:23.358417 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:23.402782 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bdd7befef39489a88a9703bc42fc1ce'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:23.403473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bdd7befef39489a88a9703bc42fc1ce'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:23.410512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bdd7befef39489a88a9703bc42fc1ce'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:23.411167 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bdd7befef39489a88a9703bc42fc1ce'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:23.423510 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bdd7befef39489a88a9703bc42fc1ce'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:23.436466 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1395/2807] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:23 2022] DELETE /identity/v3/users/3bdd7befef39489a88a9703bc42fc1ce => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:23.441021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd81a1-d817-4d43-b7f5-125198e60460 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:23.441333 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd81a1-d817-4d43-b7f5-125198e60460 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:23.441610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acd81a1-d817-4d43-b7f5-125198e60460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:23.460105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-2acd81a1-d817-4d43-b7f5-125198e60460 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:23.461336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2acd81a1-d817-4d43-b7f5-125198e60460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:23.518884 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1413/2808] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:23.528632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:23.530333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:23.571384 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:23.590574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019d16fd-1937-48e4-9304-7b6cdefe062f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:23.594723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a18ec-9953-42f5-8d3c-7698500bb148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:23.595318 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a18ec-9953-42f5-8d3c-7698500bb148 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:23.595599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a18ec-9953-42f5-8d3c-7698500bb148 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:23.595870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a18ec-9953-42f5-8d3c-7698500bb148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:23.604809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a18ec-9953-42f5-8d3c-7698500bb148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:23.606269 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a18ec-9953-42f5-8d3c-7698500bb148 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:23.616414 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1396/2809] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:23.881771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd81a1-d817-4d43-b7f5-125198e60460 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:23.883257 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2acd81a1-d817-4d43-b7f5-125198e60460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:23.895612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acd81a1-d817-4d43-b7f5-125198e60460 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:23.900604 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:23.901272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:23.901556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:23.901830 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] PATH_INFO: `/v3/projects/960cda3cd79c44dabf1748e44118aeb6` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:23.902963 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=960cda3cd79c44dabf1748e44118aeb6)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:23.905958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:23.953369 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '960cda3cd79c44dabf1748e44118aeb6', 'request_id': 'req-65944b03-ba0a-4b85-9045-59d3f8e336ee'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:23.959682 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '960cda3cd79c44dabf1748e44118aeb6', 'request_id': 'req-65944b03-ba0a-4b85-9045-59d3f8e336ee'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:23.965323 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '960cda3cd79c44dabf1748e44118aeb6', 'request_id': 'req-65944b03-ba0a-4b85-9045-59d3f8e336ee'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:23.980111 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] The token cache is being invalidate because project 960cda3cd79c44dabf1748e44118aeb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:23.980408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:23.981248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:23.983767 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1414/2810] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:23 2022] DELETE /identity/v3/projects/960cda3cd79c44dabf1748e44118aeb6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:23.987884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a18ec-9953-42f5-8d3c-7698500bb148 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:23.997495 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a18ec-9953-42f5-8d3c-7698500bb148 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:24.051725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff424d9a-45e4-471e-9d8e-2470aafcdbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:24.052349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff424d9a-45e4-471e-9d8e-2470aafcdbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:24.052624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff424d9a-45e4-471e-9d8e-2470aafcdbad admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:24.052891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff424d9a-45e4-471e-9d8e-2470aafcdbad admin admin] PATH_INFO: `/v3/users/f8efdb7f24fd452e868a161ddbc1839f` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:24.054040 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff424d9a-45e4-471e-9d8e-2470aafcdbad admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8efdb7f24fd452e868a161ddbc1839f)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:24.055984 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff424d9a-45e4-471e-9d8e-2470aafcdbad admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:24.103020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ff424d9a-45e4-471e-9d8e-2470aafcdbad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8efdb7f24fd452e868a161ddbc1839f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:24.103664 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ff424d9a-45e4-471e-9d8e-2470aafcdbad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8efdb7f24fd452e868a161ddbc1839f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:24.110651 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ff424d9a-45e4-471e-9d8e-2470aafcdbad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8efdb7f24fd452e868a161ddbc1839f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:24.111276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ff424d9a-45e4-471e-9d8e-2470aafcdbad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8efdb7f24fd452e868a161ddbc1839f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:24.123406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-ff424d9a-45e4-471e-9d8e-2470aafcdbad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8efdb7f24fd452e868a161ddbc1839f'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:24.136499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1397/2811] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:23 2022] DELETE /identity/v3/users/f8efdb7f24fd452e868a161ddbc1839f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:24.404871 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:24.406398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:24.419888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65944b03-ba0a-4b85-9045-59d3f8e336ee admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:24.465410 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:24.479398 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:24.480033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:24.480311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:24.480589 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] PATH_INFO: `/v3/projects/fecea78a61954c8e8caad253a02f98a3` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:24.481797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fecea78a61954c8e8caad253a02f98a3)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:24.483693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:24.529188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fecea78a61954c8e8caad253a02f98a3', 'request_id': 'req-9c173e9b-3a61-4251-9d9e-42a472a25c84'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:24.535312 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fecea78a61954c8e8caad253a02f98a3', 'request_id': 'req-9c173e9b-3a61-4251-9d9e-42a472a25c84'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:24.540817 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fecea78a61954c8e8caad253a02f98a3', 'request_id': 'req-9c173e9b-3a61-4251-9d9e-42a472a25c84'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:24.553006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] The token cache is being invalidate because project fecea78a61954c8e8caad253a02f98a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:24.553311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:24.553930 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9c173e9b-3a61-4251-9d9e-42a472a25c84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:24.555400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1415/2812] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:24 2022] DELETE /identity/v3/projects/fecea78a61954c8e8caad253a02f98a3 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 22 22:26:24.560878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4cc75-b641-47bb-afb0-f8cae44c0b87 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:24.561255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4cc75-b641-47bb-afb0-f8cae44c0b87 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:24.561529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4cc75-b641-47bb-afb0-f8cae44c0b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:24.581186 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2ab4cc75-b641-47bb-afb0-f8cae44c0b87 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:24.582321 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2ab4cc75-b641-47bb-afb0-f8cae44c0b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:24.632079 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:24.644402 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1398/2813] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:24.652245 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769ca5f-cfdd-4016-816d-b5faccbd342a None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:24.652533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769ca5f-cfdd-4016-816d-b5faccbd342a None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:24.652809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769ca5f-cfdd-4016-816d-b5faccbd342a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:24.673072 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d769ca5f-cfdd-4016-816d-b5faccbd342a None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:24.674299 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d769ca5f-cfdd-4016-816d-b5faccbd342a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:24.682149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1416/2814] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:24.687112 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4cc75-b641-47bb-afb0-f8cae44c0b87 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:24.696736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4cc75-b641-47bb-afb0-f8cae44c0b87 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:24.702374 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cfda0-8f3d-49f1-800d-c56be0594708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:24.702963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cfda0-8f3d-49f1-800d-c56be0594708 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:24.703241 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cfda0-8f3d-49f1-800d-c56be0594708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:24.703512 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220cfda0-8f3d-49f1-800d-c56be0594708 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:24.704150 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cfda0-8f3d-49f1-800d-c56be0594708 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:26:24.704452 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cfda0-8f3d-49f1-800d-c56be0594708 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:24.706076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220cfda0-8f3d-49f1-800d-c56be0594708 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:24.717915 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1399/2815] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:26:24.722083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769ca5f-cfdd-4016-816d-b5faccbd342a None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:24.731200 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769ca5f-cfdd-4016-816d-b5faccbd342a None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:24.735915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef6a0b-ecdb-4b64-b77c-bd1ededf59f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:24.736504 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef6a0b-ecdb-4b64-b77c-bd1ededf59f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:24.736780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef6a0b-ecdb-4b64-b77c-bd1ededf59f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:24.737052 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef6a0b-ecdb-4b64-b77c-bd1ededf59f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:24.737743 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef6a0b-ecdb-4b64-b77c-bd1ededf59f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:24.739608 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef6a0b-ecdb-4b64-b77c-bd1ededf59f0 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:24.766849 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1417/2816] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:24 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:24.770868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cfda0-8f3d-49f1-800d-c56be0594708 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:24.780680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220cfda0-8f3d-49f1-800d-c56be0594708 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:24.835610 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7afeb2-28e2-4f3c-a417-41da2b24807f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:24.836125 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7afeb2-28e2-4f3c-a417-41da2b24807f admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:24.836398 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7afeb2-28e2-4f3c-a417-41da2b24807f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:24.836667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7afeb2-28e2-4f3c-a417-41da2b24807f admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:24.837320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7afeb2-28e2-4f3c-a417-41da2b24807f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:24.839198 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7afeb2-28e2-4f3c-a417-41da2b24807f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:24.860793 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1400/2817] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:24 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:24.864842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef6a0b-ecdb-4b64-b77c-bd1ededf59f0 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:24.875643 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef6a0b-ecdb-4b64-b77c-bd1ededf59f0 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:24.880179 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd5d57-f2b5-49ae-a650-806cdd81122d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:24.880765 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd5d57-f2b5-49ae-a650-806cdd81122d admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:24.881039 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd5d57-f2b5-49ae-a650-806cdd81122d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:24.881362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bd5d57-f2b5-49ae-a650-806cdd81122d admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:24.881991 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd5d57-f2b5-49ae-a650-806cdd81122d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:24.883556 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bd5d57-f2b5-49ae-a650-806cdd81122d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:24.893362 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1418/2818] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:24.897170 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7afeb2-28e2-4f3c-a417-41da2b24807f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:24.907445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7afeb2-28e2-4f3c-a417-41da2b24807f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:24.911970 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f5d91-3a95-49c8-bbf5-f628378bb6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:24.912553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f5d91-3a95-49c8-bbf5-f628378bb6f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:24.912827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f5d91-3a95-49c8-bbf5-f628378bb6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:24.913096 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f5d91-3a95-49c8-bbf5-f628378bb6f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:24.913761 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f5d91-3a95-49c8-bbf5-f628378bb6f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:24.915316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f5d91-3a95-49c8-bbf5-f628378bb6f0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:24.926012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1401/2819] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:24.929940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd5d57-f2b5-49ae-a650-806cdd81122d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:24.940280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bd5d57-f2b5-49ae-a650-806cdd81122d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:24.944826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5dd7f-9983-41f0-9c50-0a0bf6b2a602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:24.945492 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5dd7f-9983-41f0-9c50-0a0bf6b2a602 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:24.945768 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5dd7f-9983-41f0-9c50-0a0bf6b2a602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:24.946037 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c5dd7f-9983-41f0-9c50-0a0bf6b2a602 admin admin] PATH_INFO: `/v3/projects/c15c3b41e9944edda2e74d26b274b79e/users/58b8ebc68b094273a27ee0b740f59895/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:24.961451 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5dd7f-9983-41f0-9c50-0a0bf6b2a602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15c3b41e9944edda2e74d26b274b79e, user_id=58b8ebc68b094273a27ee0b740f59895, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:24.964033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c5dd7f-9983-41f0-9c50-0a0bf6b2a602 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:24.976265 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1419/2820] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:24 2022] PUT /identity/v3/projects/c15c3b41e9944edda2e74d26b274b79e/users/58b8ebc68b094273a27ee0b740f59895/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:24.987586 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a176fd-89d6-4be7-93b9-7430a08e1516 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:24.987869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a176fd-89d6-4be7-93b9-7430a08e1516 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:24.988142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a176fd-89d6-4be7-93b9-7430a08e1516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:25.017191 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-85a176fd-89d6-4be7-93b9-7430a08e1516 None None] MFA Rules not processed for user `58b8ebc68b094273a27ee0b740f59895`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:25.018393 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-85a176fd-89d6-4be7-93b9-7430a08e1516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:25.083931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1402/2821] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:24 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:26:25.088429 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5dd7f-9983-41f0-9c50-0a0bf6b2a602 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:25.148812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c5dd7f-9983-41f0-9c50-0a0bf6b2a602 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:25.154055 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bc0d3-0200-457a-a09c-5203bf1bd00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:25.154661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bc0d3-0200-457a-a09c-5203bf1bd00e admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:25.154934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bc0d3-0200-457a-a09c-5203bf1bd00e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:25.155203 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bc0d3-0200-457a-a09c-5203bf1bd00e admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:25.155845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bc0d3-0200-457a-a09c-5203bf1bd00e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:25.157752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0bc0d3-0200-457a-a09c-5203bf1bd00e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:25.177668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:25.185856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1420/2822] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:25 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:25.189662 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a176fd-89d6-4be7-93b9-7430a08e1516 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:25.200250 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a176fd-89d6-4be7-93b9-7430a08e1516 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:25.247825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:25.275370 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6df8d4-09ee-4c65-a728-868a23b6c0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:25.275998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6df8d4-09ee-4c65-a728-868a23b6c0f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:25.276273 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6df8d4-09ee-4c65-a728-868a23b6c0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:25.276538 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6df8d4-09ee-4c65-a728-868a23b6c0f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:25.277208 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6df8d4-09ee-4c65-a728-868a23b6c0f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:25.279714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6df8d4-09ee-4c65-a728-868a23b6c0f0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:25.301519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1403/2823] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:25 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:25.305597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bc0d3-0200-457a-a09c-5203bf1bd00e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:25.316021 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0bc0d3-0200-457a-a09c-5203bf1bd00e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:25.320695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8a6a2-3b9d-457e-af95-94258cba92af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:25.321319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8a6a2-3b9d-457e-af95-94258cba92af admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:25.321598 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8a6a2-3b9d-457e-af95-94258cba92af admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:25.321868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8a6a2-3b9d-457e-af95-94258cba92af admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:25.322503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8a6a2-3b9d-457e-af95-94258cba92af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:25.324069 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8a6a2-3b9d-457e-af95-94258cba92af admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:25.334810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1421/2824] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:25.339072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6df8d4-09ee-4c65-a728-868a23b6c0f0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:25.350188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6df8d4-09ee-4c65-a728-868a23b6c0f0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:25.354844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ec0ae-6721-494a-8851-e14da731f241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:25.355499 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ec0ae-6721-494a-8851-e14da731f241 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:25.355765 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ec0ae-6721-494a-8851-e14da731f241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:25.356039 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ec0ae-6721-494a-8851-e14da731f241 admin admin] PATH_INFO: `/v3/projects/e6efe761763548c1b08e311a0e9cc606/users/3102a828e6af4948af533a8a838728d4/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:25.371334 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ec0ae-6721-494a-8851-e14da731f241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6efe761763548c1b08e311a0e9cc606, user_id=3102a828e6af4948af533a8a838728d4, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:25.374298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40ec0ae-6721-494a-8851-e14da731f241 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:25.387113 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1404/2825] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:25 2022] PUT /identity/v3/projects/e6efe761763548c1b08e311a0e9cc606/users/3102a828e6af4948af533a8a838728d4/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:25.391298 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8a6a2-3b9d-457e-af95-94258cba92af admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:25.401271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8a6a2-3b9d-457e-af95-94258cba92af admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:25.457550 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31317b9-ac8f-4169-991a-71db872af07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:25.458587 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31317b9-ac8f-4169-991a-71db872af07a admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:25.458989 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31317b9-ac8f-4169-991a-71db872af07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:25.459399 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31317b9-ac8f-4169-991a-71db872af07a admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:25.460044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31317b9-ac8f-4169-991a-71db872af07a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:25.461669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31317b9-ac8f-4169-991a-71db872af07a admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:25.471950 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1422/2826] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:25.476289 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ec0ae-6721-494a-8851-e14da731f241 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:25.487429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40ec0ae-6721-494a-8851-e14da731f241 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:25.492019 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad1515-e0aa-4505-a387-b83c402438a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:25.492656 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad1515-e0aa-4505-a387-b83c402438a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:25.492932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad1515-e0aa-4505-a387-b83c402438a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:25.493228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad1515-e0aa-4505-a387-b83c402438a1 admin admin] PATH_INFO: `/v3/projects/e6efe761763548c1b08e311a0e9cc606/users/3102a828e6af4948af533a8a838728d4/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:25.495124 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad1515-e0aa-4505-a387-b83c402438a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6efe761763548c1b08e311a0e9cc606, user_id=3102a828e6af4948af533a8a838728d4, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:25.497913 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad1515-e0aa-4505-a387-b83c402438a1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:25.507808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1405/2827] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:25 2022] PUT /identity/v3/projects/e6efe761763548c1b08e311a0e9cc606/users/3102a828e6af4948af533a8a838728d4/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:25.572356 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9a9e0-ac67-43f2-81d4-d3c95e568f5e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:25.572637 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9a9e0-ac67-43f2-81d4-d3c95e568f5e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:25.572911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a9a9e0-ac67-43f2-81d4-d3c95e568f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:25.602910 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-59a9a9e0-ac67-43f2-81d4-d3c95e568f5e None None] MFA Rules not processed for user `3102a828e6af4948af533a8a838728d4`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:25.604070 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-59a9a9e0-ac67-43f2-81d4-d3c95e568f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:25.671177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1423/2828] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:25 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:25.682335 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad1515-e0aa-4505-a387-b83c402438a1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:25.684075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-79ad1515-e0aa-4505-a387-b83c402438a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:25.741156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad1515-e0aa-4505-a387-b83c402438a1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:25.744794 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e538b2-db4a-49cd-b4bf-ef292a31c40f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:25.745409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e538b2-db4a-49cd-b4bf-ef292a31c40f service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:25.745686 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e538b2-db4a-49cd-b4bf-ef292a31c40f service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:25.745956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e538b2-db4a-49cd-b4bf-ef292a31c40f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:25.756351 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e538b2-db4a-49cd-b4bf-ef292a31c40f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:25.757837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e538b2-db4a-49cd-b4bf-ef292a31c40f service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:25.768173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1406/2829] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:25 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:25.832622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9a9e0-ac67-43f2-81d4-d3c95e568f5e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:25.897086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a9a9e0-ac67-43f2-81d4-d3c95e568f5e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:25.901646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:25.902283 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:25.902563 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:25.902836 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] PATH_INFO: `/v3/users/58b8ebc68b094273a27ee0b740f59895` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:25.904230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=58b8ebc68b094273a27ee0b740f59895)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:25.906173 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:25.916843 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:25.956263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58b8ebc68b094273a27ee0b740f59895'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:25.956914 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58b8ebc68b094273a27ee0b740f59895'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:25.957548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58b8ebc68b094273a27ee0b740f59895'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:25.969853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58b8ebc68b094273a27ee0b740f59895'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:25.976648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58b8ebc68b094273a27ee0b740f59895'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:25.990232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1424/2830] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:25 2022] DELETE /identity/v3/users/58b8ebc68b094273a27ee0b740f59895 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:25.995326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45d72c-b403-4886-955d-558a6c465f57 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:25.995831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45d72c-b403-4886-955d-558a6c465f57 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:25.995939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45d72c-b403-4886-955d-558a6c465f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:26.015678 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-9b45d72c-b403-4886-955d-558a6c465f57 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:26.016882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9b45d72c-b403-4886-955d-558a6c465f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:26.084806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1407/2831] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:26.094208 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:26.095633 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:26.158100 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8986e2-55bb-436f-b440-9de9f86c22ad admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:26.161634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1adc8-32e1-4827-bd0e-22aa6ed5b530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:26.162233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1adc8-32e1-4827-bd0e-22aa6ed5b530 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:26.162510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1adc8-32e1-4827-bd0e-22aa6ed5b530 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:26.162782 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e1adc8-32e1-4827-bd0e-22aa6ed5b530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:26.171948 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1adc8-32e1-4827-bd0e-22aa6ed5b530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:26.173383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e1adc8-32e1-4827-bd0e-22aa6ed5b530 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:26.182790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1425/2832] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:26.456875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45d72c-b403-4886-955d-558a6c465f57 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:26.458591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-9b45d72c-b403-4886-955d-558a6c465f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:26.472182 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45d72c-b403-4886-955d-558a6c465f57 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:26.476727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:26.477392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:26.477667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:26.478457 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] PATH_INFO: `/v3/projects/c15c3b41e9944edda2e74d26b274b79e` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:26.479748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15c3b41e9944edda2e74d26b274b79e)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:26.482222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:26.525149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:26.529333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15c3b41e9944edda2e74d26b274b79e', 'request_id': 'req-34d307d6-c945-4c19-adf0-86a3bd89b062'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:26.536354 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15c3b41e9944edda2e74d26b274b79e', 'request_id': 'req-34d307d6-c945-4c19-adf0-86a3bd89b062'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:26.542574 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15c3b41e9944edda2e74d26b274b79e', 'request_id': 'req-34d307d6-c945-4c19-adf0-86a3bd89b062'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:26.553549 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] The token cache is being invalidate because project c15c3b41e9944edda2e74d26b274b79e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:26.553837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:26.554467 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:26.555943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1408/2833] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:26 2022] DELETE /identity/v3/projects/c15c3b41e9944edda2e74d26b274b79e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:26.560612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1adc8-32e1-4827-bd0e-22aa6ed5b530 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:26.570232 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e1adc8-32e1-4827-bd0e-22aa6ed5b530 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:26.626097 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde2ac3e-bd11-4a63-9e71-15382ce0af13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:26.626713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2ac3e-bd11-4a63-9e71-15382ce0af13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:26.626986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2ac3e-bd11-4a63-9e71-15382ce0af13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:26.627260 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde2ac3e-bd11-4a63-9e71-15382ce0af13 admin admin] PATH_INFO: `/v3/users/3102a828e6af4948af533a8a838728d4` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:26.628366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2ac3e-bd11-4a63-9e71-15382ce0af13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3102a828e6af4948af533a8a838728d4)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:26.630282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde2ac3e-bd11-4a63-9e71-15382ce0af13 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:26.663441 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:26.677289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bde2ac3e-bd11-4a63-9e71-15382ce0af13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3102a828e6af4948af533a8a838728d4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:26.677936 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bde2ac3e-bd11-4a63-9e71-15382ce0af13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3102a828e6af4948af533a8a838728d4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:26.678542 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bde2ac3e-bd11-4a63-9e71-15382ce0af13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3102a828e6af4948af533a8a838728d4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:26.690336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bde2ac3e-bd11-4a63-9e71-15382ce0af13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3102a828e6af4948af533a8a838728d4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:26.697068 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-bde2ac3e-bd11-4a63-9e71-15382ce0af13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3102a828e6af4948af533a8a838728d4'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:26.710811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1426/2834] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:26 2022] DELETE /identity/v3/users/3102a828e6af4948af533a8a838728d4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:26.976482 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:26.977975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:26.993001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d307d6-c945-4c19-adf0-86a3bd89b062 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:27.050520 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:27.051190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.051421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.051697 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] PATH_INFO: `/v3/projects/e6efe761763548c1b08e311a0e9cc606` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.052827 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6efe761763548c1b08e311a0e9cc606)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:27.054841 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:27.101518 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6efe761763548c1b08e311a0e9cc606', 'request_id': 'req-a06a2393-0e27-470c-92d6-fe93da94386c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:27.108748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6efe761763548c1b08e311a0e9cc606', 'request_id': 'req-a06a2393-0e27-470c-92d6-fe93da94386c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:27.114952 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6efe761763548c1b08e311a0e9cc606', 'request_id': 'req-a06a2393-0e27-470c-92d6-fe93da94386c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:27.125352 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] The token cache is being invalidate because project e6efe761763548c1b08e311a0e9cc606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:27.125645 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:27.126272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-a06a2393-0e27-470c-92d6-fe93da94386c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:27.127731 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1409/2835] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:26 2022] DELETE /identity/v3/projects/e6efe761763548c1b08e311a0e9cc606 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:27.132533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765de54d-7f6e-4ce4-a6c7-b316bfd212c9 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.133042 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765de54d-7f6e-4ce4-a6c7-b316bfd212c9 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.133197 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765de54d-7f6e-4ce4-a6c7-b316bfd212c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.153282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-765de54d-7f6e-4ce4-a6c7-b316bfd212c9 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:27.155288 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-765de54d-7f6e-4ce4-a6c7-b316bfd212c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:27.214121 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1427/2836] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:27.222529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcbcef-308a-4bc0-9c6c-7196accf9bf1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.222758 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcbcef-308a-4bc0-9c6c-7196accf9bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.223037 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dcbcef-308a-4bc0-9c6c-7196accf9bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.242508 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-d4dcbcef-308a-4bc0-9c6c-7196accf9bf1 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:27.243708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d4dcbcef-308a-4bc0-9c6c-7196accf9bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:27.251918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1410/2837] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:27.256044 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765de54d-7f6e-4ce4-a6c7-b316bfd212c9 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:27.266422 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765de54d-7f6e-4ce4-a6c7-b316bfd212c9 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:27.270983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b211d-1f48-4c4c-be23-f21a70372c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:27.271592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b211d-1f48-4c4c-be23-f21a70372c26 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.271850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b211d-1f48-4c4c-be23-f21a70372c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.272117 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b211d-1f48-4c4c-be23-f21a70372c26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.272751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b211d-1f48-4c4c-be23-f21a70372c26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:26:27.273050 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b211d-1f48-4c4c-be23-f21a70372c26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:27.274731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b211d-1f48-4c4c-be23-f21a70372c26 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:27.286720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1428/2838] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:26:27.291013 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcbcef-308a-4bc0-9c6c-7196accf9bf1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:27.300646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dcbcef-308a-4bc0-9c6c-7196accf9bf1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:27.305363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2ffd2-d0f7-41c3-bafa-dfdaa9fee2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:27.305954 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ffd2-d0f7-41c3-bafa-dfdaa9fee2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.306233 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ffd2-d0f7-41c3-bafa-dfdaa9fee2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.306504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2ffd2-d0f7-41c3-bafa-dfdaa9fee2a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.307142 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2ffd2-d0f7-41c3-bafa-dfdaa9fee2a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:27.309023 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2ffd2-d0f7-41c3-bafa-dfdaa9fee2a7 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:27.338010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1411/2839] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:27 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:27.342759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b211d-1f48-4c4c-be23-f21a70372c26 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:27.352906 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b211d-1f48-4c4c-be23-f21a70372c26 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:27.409942 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b613c1-f814-4eb1-b1b4-fa65242c723e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:27.410548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b613c1-f814-4eb1-b1b4-fa65242c723e admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.410823 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b613c1-f814-4eb1-b1b4-fa65242c723e admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.411094 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b613c1-f814-4eb1-b1b4-fa65242c723e admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.411726 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b613c1-f814-4eb1-b1b4-fa65242c723e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:27.413685 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b613c1-f814-4eb1-b1b4-fa65242c723e admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:27.435452 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1429/2840] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:27 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:27.439727 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2ffd2-d0f7-41c3-bafa-dfdaa9fee2a7 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:27.451501 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2ffd2-d0f7-41c3-bafa-dfdaa9fee2a7 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:27.456431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936b716-299f-4121-8b28-f8bff13418bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:27.457024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936b716-299f-4121-8b28-f8bff13418bd admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.457336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936b716-299f-4121-8b28-f8bff13418bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.457624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936b716-299f-4121-8b28-f8bff13418bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.458513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936b716-299f-4121-8b28-f8bff13418bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:27.460103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936b716-299f-4121-8b28-f8bff13418bd admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:27.470449 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1412/2841] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:27.474426 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b613c1-f814-4eb1-b1b4-fa65242c723e admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:27.484506 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b613c1-f814-4eb1-b1b4-fa65242c723e admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:27.489756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41baeb3-6cb6-48ee-81ec-e91566750df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:27.490716 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41baeb3-6cb6-48ee-81ec-e91566750df9 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.491271 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41baeb3-6cb6-48ee-81ec-e91566750df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.491581 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41baeb3-6cb6-48ee-81ec-e91566750df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.492274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41baeb3-6cb6-48ee-81ec-e91566750df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:27.494408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41baeb3-6cb6-48ee-81ec-e91566750df9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:27.505907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1430/2842] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:27.510083 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936b716-299f-4121-8b28-f8bff13418bd admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:27.520708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936b716-299f-4121-8b28-f8bff13418bd admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:27.525365 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47ebd7-18b8-4a4a-9431-53cad02dcc42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:27.525998 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47ebd7-18b8-4a4a-9431-53cad02dcc42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.526271 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47ebd7-18b8-4a4a-9431-53cad02dcc42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.526541 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47ebd7-18b8-4a4a-9431-53cad02dcc42 admin admin] PATH_INFO: `/v3/projects/7efbe567e67f43efbe183106d5620558/users/1943835a1ba54ef49312d9ca67aa2a85/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.540815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47ebd7-18b8-4a4a-9431-53cad02dcc42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7efbe567e67f43efbe183106d5620558, user_id=1943835a1ba54ef49312d9ca67aa2a85, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:27.543443 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47ebd7-18b8-4a4a-9431-53cad02dcc42 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:27.550947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:27.556695 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1413/2843] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:27 2022] PUT /identity/v3/projects/7efbe567e67f43efbe183106d5620558/users/1943835a1ba54ef49312d9ca67aa2a85/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:27.567574 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd09ca-e59d-4db3-ad8e-4d69d3da5be0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.567855 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd09ca-e59d-4db3-ad8e-4d69d3da5be0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.568125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd09ca-e59d-4db3-ad8e-4d69d3da5be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.601287 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-79bd09ca-e59d-4db3-ad8e-4d69d3da5be0 None None] MFA Rules not processed for user `1943835a1ba54ef49312d9ca67aa2a85`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:27.602526 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-79bd09ca-e59d-4db3-ad8e-4d69d3da5be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:27.606600 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:27.672335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1431/2844] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:27.676868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47ebd7-18b8-4a4a-9431-53cad02dcc42 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:27.736560 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47ebd7-18b8-4a4a-9431-53cad02dcc42 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:27.741037 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600cbf0-0d31-4fdd-901c-6b1bfa20afda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:27.741647 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600cbf0-0d31-4fdd-901c-6b1bfa20afda admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.741920 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600cbf0-0d31-4fdd-901c-6b1bfa20afda admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.742188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600cbf0-0d31-4fdd-901c-6b1bfa20afda admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.742815 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600cbf0-0d31-4fdd-901c-6b1bfa20afda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:27.744676 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600cbf0-0d31-4fdd-901c-6b1bfa20afda admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:27.771591 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1414/2845] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:27 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:27.775801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd09ca-e59d-4db3-ad8e-4d69d3da5be0 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:27.786762 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd09ca-e59d-4db3-ad8e-4d69d3da5be0 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:27.842720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17874225-af5b-45a1-8a9a-c554e0bcd003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:27.843321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874225-af5b-45a1-8a9a-c554e0bcd003 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.843597 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874225-af5b-45a1-8a9a-c554e0bcd003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.843865 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17874225-af5b-45a1-8a9a-c554e0bcd003 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.844482 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17874225-af5b-45a1-8a9a-c554e0bcd003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:27.846423 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17874225-af5b-45a1-8a9a-c554e0bcd003 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:27.867015 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1432/2846] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:27 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:26:27.871020 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600cbf0-0d31-4fdd-901c-6b1bfa20afda admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:27.881813 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600cbf0-0d31-4fdd-901c-6b1bfa20afda admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:27.886433 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737bcbe-86d0-4b60-bd12-f2e5dff72008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:27.887017 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737bcbe-86d0-4b60-bd12-f2e5dff72008 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.887287 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737bcbe-86d0-4b60-bd12-f2e5dff72008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.887553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737bcbe-86d0-4b60-bd12-f2e5dff72008 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.888379 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737bcbe-86d0-4b60-bd12-f2e5dff72008 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:27.889994 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737bcbe-86d0-4b60-bd12-f2e5dff72008 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:27.900173 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1415/2847] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:27.904337 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17874225-af5b-45a1-8a9a-c554e0bcd003 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:27.914510 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17874225-af5b-45a1-8a9a-c554e0bcd003 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:27.920104 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fb8c7-b24c-451d-93f3-af4671dcb0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:27.920378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb8c7-b24c-451d-93f3-af4671dcb0b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:27.920653 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb8c7-b24c-451d-93f3-af4671dcb0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:27.920926 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fb8c7-b24c-451d-93f3-af4671dcb0b3 admin admin] PATH_INFO: `/v3/projects/721bc4d1ec10455a867533fa363f2697/users/2d2ba48b187b4ba59374601fd15ce050/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:27.941965 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fb8c7-b24c-451d-93f3-af4671dcb0b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=721bc4d1ec10455a867533fa363f2697, user_id=2d2ba48b187b4ba59374601fd15ce050, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:27.944582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fb8c7-b24c-451d-93f3-af4671dcb0b3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:27.958769 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1433/2848] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:27 2022] PUT /identity/v3/projects/721bc4d1ec10455a867533fa363f2697/users/2d2ba48b187b4ba59374601fd15ce050/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:27.962935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737bcbe-86d0-4b60-bd12-f2e5dff72008 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:27.973065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737bcbe-86d0-4b60-bd12-f2e5dff72008 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:28.028674 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5403b0e7-f752-476d-b412-ae842db9b2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:28.029445 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403b0e7-f752-476d-b412-ae842db9b2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:28.029723 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403b0e7-f752-476d-b412-ae842db9b2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:28.029991 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403b0e7-f752-476d-b412-ae842db9b2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:28.030623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403b0e7-f752-476d-b412-ae842db9b2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:28.032199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5403b0e7-f752-476d-b412-ae842db9b2a2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:28.041668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1416/2849] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:28.046335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fb8c7-b24c-451d-93f3-af4671dcb0b3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:28.056468 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fb8c7-b24c-451d-93f3-af4671dcb0b3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:28.061885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d09d35-cb0d-49b2-89c3-0a2eb6814d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:28.062701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d09d35-cb0d-49b2-89c3-0a2eb6814d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:28.063081 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d09d35-cb0d-49b2-89c3-0a2eb6814d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:28.063416 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d09d35-cb0d-49b2-89c3-0a2eb6814d5f admin admin] PATH_INFO: `/v3/projects/721bc4d1ec10455a867533fa363f2697/users/2d2ba48b187b4ba59374601fd15ce050/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:28.065472 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d09d35-cb0d-49b2-89c3-0a2eb6814d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=721bc4d1ec10455a867533fa363f2697, user_id=2d2ba48b187b4ba59374601fd15ce050, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:28.068140 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d09d35-cb0d-49b2-89c3-0a2eb6814d5f admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:28.078886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1434/2850] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:28 2022] PUT /identity/v3/projects/721bc4d1ec10455a867533fa363f2697/users/2d2ba48b187b4ba59374601fd15ce050/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:28.090302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b29fa-a6e3-4376-87a5-227ee9a7668b None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:28.090774 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b29fa-a6e3-4376-87a5-227ee9a7668b None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:28.090898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b29fa-a6e3-4376-87a5-227ee9a7668b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:28.117859 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0c8b29fa-a6e3-4376-87a5-227ee9a7668b None None] MFA Rules not processed for user `2d2ba48b187b4ba59374601fd15ce050`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:28.119044 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0c8b29fa-a6e3-4376-87a5-227ee9a7668b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:28.178593 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:28.189889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1417/2851] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:28 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:28.201420 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d09d35-cb0d-49b2-89c3-0a2eb6814d5f admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:28.203198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-53d09d35-cb0d-49b2-89c3-0a2eb6814d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:28.261485 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d09d35-cb0d-49b2-89c3-0a2eb6814d5f admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:28.265360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d916fb8-0495-48d0-9748-b977022e5596 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:28.265967 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d916fb8-0495-48d0-9748-b977022e5596 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:28.266237 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d916fb8-0495-48d0-9748-b977022e5596 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:28.266503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d916fb8-0495-48d0-9748-b977022e5596 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:28.275655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d916fb8-0495-48d0-9748-b977022e5596 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:28.277063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d916fb8-0495-48d0-9748-b977022e5596 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:28.284612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:28.290126 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1435/2852] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:28.393800 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b29fa-a6e3-4376-87a5-227ee9a7668b None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:28.403410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b29fa-a6e3-4376-87a5-227ee9a7668b None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:28.407667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf4ea8-d75a-4bcd-b787-b0278f690e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:28.408255 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4ea8-d75a-4bcd-b787-b0278f690e7e service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:28.408533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4ea8-d75a-4bcd-b787-b0278f690e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:28.408806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf4ea8-d75a-4bcd-b787-b0278f690e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:28.463975 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf4ea8-d75a-4bcd-b787-b0278f690e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:28.465436 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bf4ea8-d75a-4bcd-b787-b0278f690e7e service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:28.474850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1418/2853] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:31.060861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d916fb8-0495-48d0-9748-b977022e5596 service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:31.123425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d916fb8-0495-48d0-9748-b977022e5596 service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:31.127880 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:31.128624 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:31.128902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:31.129202 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] PATH_INFO: `/v3/users/1943835a1ba54ef49312d9ca67aa2a85` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:31.130301 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1943835a1ba54ef49312d9ca67aa2a85)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:31.132184 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:31.180086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1943835a1ba54ef49312d9ca67aa2a85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.180754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1943835a1ba54ef49312d9ca67aa2a85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.181386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1943835a1ba54ef49312d9ca67aa2a85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.193761 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1943835a1ba54ef49312d9ca67aa2a85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.201092 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1943835a1ba54ef49312d9ca67aa2a85'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.214014 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1436/2854] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:31 2022] DELETE /identity/v3/users/1943835a1ba54ef49312d9ca67aa2a85 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:31.218620 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2113a4-2a92-4873-8b3c-f8ed8d9bcfd3 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:31.219114 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2113a4-2a92-4873-8b3c-f8ed8d9bcfd3 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:31.219221 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2113a4-2a92-4873-8b3c-f8ed8d9bcfd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:31.237635 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-0f2113a4-2a92-4873-8b3c-f8ed8d9bcfd3 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:31.238824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0f2113a4-2a92-4873-8b3c-f8ed8d9bcfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:31.296088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1419/2855] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:31.305717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:31.307141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:31.364053 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b28c7-4649-4a54-87c6-12457576eeac admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:31.367695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2df9d-29b0-423c-89fe-c37bc6c4b476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:31.368293 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2df9d-29b0-423c-89fe-c37bc6c4b476 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:31.368569 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2df9d-29b0-423c-89fe-c37bc6c4b476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:31.368839 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2df9d-29b0-423c-89fe-c37bc6c4b476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:31.378439 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2df9d-29b0-423c-89fe-c37bc6c4b476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:31.380000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b2df9d-29b0-423c-89fe-c37bc6c4b476 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:31.389866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1437/2856] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:31.651605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2113a4-2a92-4873-8b3c-f8ed8d9bcfd3 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:31.653155 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-0f2113a4-2a92-4873-8b3c-f8ed8d9bcfd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:31.665638 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2113a4-2a92-4873-8b3c-f8ed8d9bcfd3 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:31.670918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:31.671616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:31.671891 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:31.672160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] PATH_INFO: `/v3/projects/7efbe567e67f43efbe183106d5620558` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:31.673297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7efbe567e67f43efbe183106d5620558)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:31.675247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:31.721312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7efbe567e67f43efbe183106d5620558', 'request_id': 'req-2efca2cd-18af-4838-ad94-e2e88943cf9e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.728507 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7efbe567e67f43efbe183106d5620558', 'request_id': 'req-2efca2cd-18af-4838-ad94-e2e88943cf9e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.735330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7efbe567e67f43efbe183106d5620558', 'request_id': 'req-2efca2cd-18af-4838-ad94-e2e88943cf9e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.745581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] The token cache is being invalidate because project 7efbe567e67f43efbe183106d5620558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:31.745867 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.746493 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.748597 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1420/2857] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:31 2022] DELETE /identity/v3/projects/7efbe567e67f43efbe183106d5620558 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:31.752708 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2df9d-29b0-423c-89fe-c37bc6c4b476 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:31.762105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2df9d-29b0-423c-89fe-c37bc6c4b476 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:31.779934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:31.820406 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7a5ea-b132-4fba-83b1-3c63afaf16f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:31.821033 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a5ea-b132-4fba-83b1-3c63afaf16f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:31.821338 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a5ea-b132-4fba-83b1-3c63afaf16f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:31.821625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a5ea-b132-4fba-83b1-3c63afaf16f3 admin admin] PATH_INFO: `/v3/users/2d2ba48b187b4ba59374601fd15ce050` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:31.822721 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7a5ea-b132-4fba-83b1-3c63afaf16f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d2ba48b187b4ba59374601fd15ce050)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:31.824607 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7a5ea-b132-4fba-83b1-3c63afaf16f3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:31.870355 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17c7a5ea-b132-4fba-83b1-3c63afaf16f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d2ba48b187b4ba59374601fd15ce050'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.870928 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17c7a5ea-b132-4fba-83b1-3c63afaf16f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d2ba48b187b4ba59374601fd15ce050'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.871524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17c7a5ea-b132-4fba-83b1-3c63afaf16f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d2ba48b187b4ba59374601fd15ce050'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.883735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17c7a5ea-b132-4fba-83b1-3c63afaf16f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d2ba48b187b4ba59374601fd15ce050'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.890570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-17c7a5ea-b132-4fba-83b1-3c63afaf16f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d2ba48b187b4ba59374601fd15ce050'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:31.902717 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1438/2858] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:31 2022] DELETE /identity/v3/users/2d2ba48b187b4ba59374601fd15ce050 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:32.185995 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:32.187682 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:32.201491 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efca2cd-18af-4838-ad94-e2e88943cf9e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:32.214531 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:32.260024 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:32.260695 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:32.260912 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:32.261223 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] PATH_INFO: `/v3/projects/721bc4d1ec10455a867533fa363f2697` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:32.262337 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] RBAC: Authorizing `identity:delete_project(project_id=721bc4d1ec10455a867533fa363f2697)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:32.264767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:32.310956 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '721bc4d1ec10455a867533fa363f2697', 'request_id': 'req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:32.317900 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '721bc4d1ec10455a867533fa363f2697', 'request_id': 'req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:32.324032 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '721bc4d1ec10455a867533fa363f2697', 'request_id': 'req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:32.334683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] The token cache is being invalidate because project 721bc4d1ec10455a867533fa363f2697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:32.334965 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:32.335605 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-6f6d5fc2-8496-4bfb-b16a-c642adb6b14c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:32.337062 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1421/2859] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:32 2022] DELETE /identity/v3/projects/721bc4d1ec10455a867533fa363f2697 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:32.342189 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afe06c-476e-401f-80ec-f67a1edfd602 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:32.342679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afe06c-476e-401f-80ec-f67a1edfd602 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:32.342787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66afe06c-476e-401f-80ec-f67a1edfd602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:32.362378 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-66afe06c-476e-401f-80ec-f67a1edfd602 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:32.363572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-66afe06c-476e-401f-80ec-f67a1edfd602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:32.421186 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1439/2860] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:32.428660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269426d-44c1-4e3b-b636-f2a09d53d45f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:32.428943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269426d-44c1-4e3b-b636-f2a09d53d45f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:32.429311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269426d-44c1-4e3b-b636-f2a09d53d45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:32.449159 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-2269426d-44c1-4e3b-b636-f2a09d53d45f None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:32.450335 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-2269426d-44c1-4e3b-b636-f2a09d53d45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:32.458195 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1422/2861] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:32.462481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afe06c-476e-401f-80ec-f67a1edfd602 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:32.472224 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66afe06c-476e-401f-80ec-f67a1edfd602 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:32.476759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8399-6da2-4226-b6ec-fbe12a54bdf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:32.477391 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8399-6da2-4226-b6ec-fbe12a54bdf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:32.477669 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8399-6da2-4226-b6ec-fbe12a54bdf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:32.477937 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c8399-6da2-4226-b6ec-fbe12a54bdf6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:32.478568 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c8399-6da2-4226-b6ec-fbe12a54bdf6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:26:32.478866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c8399-6da2-4226-b6ec-fbe12a54bdf6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:32.480503 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c8399-6da2-4226-b6ec-fbe12a54bdf6 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:32.492570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1440/2862] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:26:32.496943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269426d-44c1-4e3b-b636-f2a09d53d45f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:32.506722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269426d-44c1-4e3b-b636-f2a09d53d45f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:32.511342 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab56d2-2777-4409-83c0-ae97c6e8dc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:32.511927 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab56d2-2777-4409-83c0-ae97c6e8dc50 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:32.512203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab56d2-2777-4409-83c0-ae97c6e8dc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:32.512475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab56d2-2777-4409-83c0-ae97c6e8dc50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:32.513126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab56d2-2777-4409-83c0-ae97c6e8dc50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:32.515003 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fab56d2-2777-4409-83c0-ae97c6e8dc50 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:32.541253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1423/2863] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:32 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:32.545812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8399-6da2-4226-b6ec-fbe12a54bdf6 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:32.556915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c8399-6da2-4226-b6ec-fbe12a54bdf6 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:32.582986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:32.614190 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91914c56-0ddb-46d0-aa2f-1f67835f4005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:32.614786 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91914c56-0ddb-46d0-aa2f-1f67835f4005 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:32.615065 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91914c56-0ddb-46d0-aa2f-1f67835f4005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:32.615336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91914c56-0ddb-46d0-aa2f-1f67835f4005 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:32.615970 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91914c56-0ddb-46d0-aa2f-1f67835f4005 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:32.617902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91914c56-0ddb-46d0-aa2f-1f67835f4005 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:32.638642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1441/2864] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:32 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:26:32.642760 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab56d2-2777-4409-83c0-ae97c6e8dc50 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:32.653064 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fab56d2-2777-4409-83c0-ae97c6e8dc50 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:32.658327 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c72f3a-45c8-47fa-b505-354f973b98af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:32.658919 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c72f3a-45c8-47fa-b505-354f973b98af admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:32.659193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c72f3a-45c8-47fa-b505-354f973b98af admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:32.659464 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c72f3a-45c8-47fa-b505-354f973b98af admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:32.660103 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c72f3a-45c8-47fa-b505-354f973b98af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:32.661725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c72f3a-45c8-47fa-b505-354f973b98af admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:32.672458 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1424/2865] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:32.676295 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91914c56-0ddb-46d0-aa2f-1f67835f4005 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:32.686264 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91914c56-0ddb-46d0-aa2f-1f67835f4005 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:32.690798 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ffe582-24e2-4dec-853b-0691b938d29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:32.691376 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe582-24e2-4dec-853b-0691b938d29b admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:32.691649 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe582-24e2-4dec-853b-0691b938d29b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:32.691916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ffe582-24e2-4dec-853b-0691b938d29b admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:32.692549 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffe582-24e2-4dec-853b-0691b938d29b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:32.694137 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ffe582-24e2-4dec-853b-0691b938d29b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:32.708229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1442/2866] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:32.712363 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c72f3a-45c8-47fa-b505-354f973b98af admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:32.723188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c72f3a-45c8-47fa-b505-354f973b98af admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:32.727831 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac976305-d10a-4bf4-9956-9cb3c92a1007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:32.728470 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac976305-d10a-4bf4-9956-9cb3c92a1007 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:32.728750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac976305-d10a-4bf4-9956-9cb3c92a1007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:32.729027 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac976305-d10a-4bf4-9956-9cb3c92a1007 admin admin] PATH_INFO: `/v3/projects/1209c934a04845cabc02dd4236f28609/users/19879d9f29024916b9f9333bed5d4cd8/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:32.744097 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac976305-d10a-4bf4-9956-9cb3c92a1007 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1209c934a04845cabc02dd4236f28609, user_id=19879d9f29024916b9f9333bed5d4cd8, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:32.746756 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac976305-d10a-4bf4-9956-9cb3c92a1007 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:32.759875 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1425/2867] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:32 2022] PUT /identity/v3/projects/1209c934a04845cabc02dd4236f28609/users/19879d9f29024916b9f9333bed5d4cd8/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:32.770262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746e656-ba29-46cb-99a4-c0d21b94751e None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:32.770548 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746e656-ba29-46cb-99a4-c0d21b94751e None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:32.770821 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d746e656-ba29-46cb-99a4-c0d21b94751e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:32.799289 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d746e656-ba29-46cb-99a4-c0d21b94751e None None] MFA Rules not processed for user `19879d9f29024916b9f9333bed5d4cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:32.800500 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d746e656-ba29-46cb-99a4-c0d21b94751e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:32.865983 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1443/2868] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:32.870694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac976305-d10a-4bf4-9956-9cb3c92a1007 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:32.915312 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:32.934803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac976305-d10a-4bf4-9956-9cb3c92a1007 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:32.941082 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d526e3a-372d-46af-966c-ec77fd447c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:32.941706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d526e3a-372d-46af-966c-ec77fd447c2f admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:32.941987 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d526e3a-372d-46af-966c-ec77fd447c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:32.942261 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d526e3a-372d-46af-966c-ec77fd447c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:32.942895 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d526e3a-372d-46af-966c-ec77fd447c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:32.944808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d526e3a-372d-46af-966c-ec77fd447c2f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:32.971653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1426/2869] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:32 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:32.975883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746e656-ba29-46cb-99a4-c0d21b94751e None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:32.987180 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d746e656-ba29-46cb-99a4-c0d21b94751e None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:33.042176 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0827dd-dce3-47fe-b4a8-79d061fe0295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:33.042780 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0827dd-dce3-47fe-b4a8-79d061fe0295 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:33.043058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0827dd-dce3-47fe-b4a8-79d061fe0295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:33.043328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0827dd-dce3-47fe-b4a8-79d061fe0295 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:33.043954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0827dd-dce3-47fe-b4a8-79d061fe0295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:33.045957 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0827dd-dce3-47fe-b4a8-79d061fe0295 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:33.066790 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1444/2870] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:32 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:33.070896 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d526e3a-372d-46af-966c-ec77fd447c2f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:33.081947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d526e3a-372d-46af-966c-ec77fd447c2f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:33.086849 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1a2a4-cca4-4ef2-beaa-4342e87f5135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:33.087437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1a2a4-cca4-4ef2-beaa-4342e87f5135 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:33.087717 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1a2a4-cca4-4ef2-beaa-4342e87f5135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:33.087989 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1a2a4-cca4-4ef2-beaa-4342e87f5135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:33.088624 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1a2a4-cca4-4ef2-beaa-4342e87f5135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:33.090253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1a2a4-cca4-4ef2-beaa-4342e87f5135 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:33.101577 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1427/2871] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:33.105701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0827dd-dce3-47fe-b4a8-79d061fe0295 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:33.115577 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0827dd-dce3-47fe-b4a8-79d061fe0295 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:33.120115 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332eeeb9-5979-4d52-ad54-431bc98a702d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:33.120746 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332eeeb9-5979-4d52-ad54-431bc98a702d admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:33.121018 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332eeeb9-5979-4d52-ad54-431bc98a702d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:33.121328 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332eeeb9-5979-4d52-ad54-431bc98a702d admin admin] PATH_INFO: `/v3/projects/08f942bc644640b6ba566989ffec1114/users/dbd2e1a4d8df422c85d8a3b90867a9fe/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:33.135826 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332eeeb9-5979-4d52-ad54-431bc98a702d admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f942bc644640b6ba566989ffec1114, user_id=dbd2e1a4d8df422c85d8a3b90867a9fe, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:33.138473 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332eeeb9-5979-4d52-ad54-431bc98a702d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:33.151887 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1445/2872] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:33 2022] PUT /identity/v3/projects/08f942bc644640b6ba566989ffec1114/users/dbd2e1a4d8df422c85d8a3b90867a9fe/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:33.156484 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1a2a4-cca4-4ef2-beaa-4342e87f5135 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:33.166460 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1a2a4-cca4-4ef2-beaa-4342e87f5135 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:33.221802 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e9f692-27cc-47dc-9e0b-283e70c4f782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:33.222311 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f692-27cc-47dc-9e0b-283e70c4f782 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:33.222589 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f692-27cc-47dc-9e0b-283e70c4f782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:33.222860 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e9f692-27cc-47dc-9e0b-283e70c4f782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:33.223491 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9f692-27cc-47dc-9e0b-283e70c4f782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:33.225065 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e9f692-27cc-47dc-9e0b-283e70c4f782 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:33.235219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1428/2873] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:33.239449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332eeeb9-5979-4d52-ad54-431bc98a702d admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:33.244820 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:33.251925 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332eeeb9-5979-4d52-ad54-431bc98a702d admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:33.256743 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f921b-31a0-4369-9297-f64d70e51281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:33.257487 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f921b-31a0-4369-9297-f64d70e51281 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:33.257771 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f921b-31a0-4369-9297-f64d70e51281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:33.258043 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f921b-31a0-4369-9297-f64d70e51281 admin admin] PATH_INFO: `/v3/projects/08f942bc644640b6ba566989ffec1114/users/dbd2e1a4d8df422c85d8a3b90867a9fe/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:33.259956 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f921b-31a0-4369-9297-f64d70e51281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f942bc644640b6ba566989ffec1114, user_id=dbd2e1a4d8df422c85d8a3b90867a9fe, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:33.262611 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f921b-31a0-4369-9297-f64d70e51281 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:33.272736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1446/2874] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:33 2022] PUT /identity/v3/projects/08f942bc644640b6ba566989ffec1114/users/dbd2e1a4d8df422c85d8a3b90867a9fe/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:33.284299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5cec-5d18-42aa-9534-c907a5447785 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:33.284797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5cec-5d18-42aa-9534-c907a5447785 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:33.284922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e5cec-5d18-42aa-9534-c907a5447785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:33.312748 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c63e5cec-5d18-42aa-9534-c907a5447785 None None] MFA Rules not processed for user `dbd2e1a4d8df422c85d8a3b90867a9fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:33.313978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c63e5cec-5d18-42aa-9534-c907a5447785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:33.383046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1429/2875] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:33.393720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f921b-31a0-4369-9297-f64d70e51281 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:33.395491 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-4d8f921b-31a0-4369-9297-f64d70e51281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:33.452736 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f921b-31a0-4369-9297-f64d70e51281 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:33.456327 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053f48d-2869-490b-a8fb-25922f506f05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:33.456909 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053f48d-2869-490b-a8fb-25922f506f05 service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:33.457212 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053f48d-2869-490b-a8fb-25922f506f05 service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:33.457483 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053f48d-2869-490b-a8fb-25922f506f05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:33.513259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053f48d-2869-490b-a8fb-25922f506f05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:33.514916 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053f48d-2869-490b-a8fb-25922f506f05 service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:33.523475 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1447/2876] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:36.054931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e5cec-5d18-42aa-9534-c907a5447785 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:36.096554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:36.118161 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e5cec-5d18-42aa-9534-c907a5447785 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:36.122699 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:36.123326 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:36.123599 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:36.123870 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] PATH_INFO: `/v3/users/19879d9f29024916b9f9333bed5d4cd8` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:36.124982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19879d9f29024916b9f9333bed5d4cd8)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:36.126939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:36.173639 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19879d9f29024916b9f9333bed5d4cd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.174308 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19879d9f29024916b9f9333bed5d4cd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.181299 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19879d9f29024916b9f9333bed5d4cd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.181943 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19879d9f29024916b9f9333bed5d4cd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.194012 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19879d9f29024916b9f9333bed5d4cd8'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.207055 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1430/2877] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:36 2022] DELETE /identity/v3/users/19879d9f29024916b9f9333bed5d4cd8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:36.211858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d4ddb-a513-4925-bc75-19164b9ce845 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:36.212338 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d4ddb-a513-4925-bc75-19164b9ce845 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:36.212439 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d4ddb-a513-4925-bc75-19164b9ce845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:36.232058 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-d41d4ddb-a513-4925-bc75-19164b9ce845 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:36.233274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d41d4ddb-a513-4925-bc75-19164b9ce845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:36.290449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1448/2878] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:36.299959 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:36.301406 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:36.357796 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8145fbed-49b5-42c6-9e03-1d9a28628795 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:36.361924 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027b120-af93-466e-8440-8382b2ab2ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:36.362525 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027b120-af93-466e-8440-8382b2ab2ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:36.362805 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027b120-af93-466e-8440-8382b2ab2ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:36.363078 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9027b120-af93-466e-8440-8382b2ab2ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:36.372183 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027b120-af93-466e-8440-8382b2ab2ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:36.373658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9027b120-af93-466e-8440-8382b2ab2ac2 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:36.383809 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1431/2879] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:36.647610 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d4ddb-a513-4925-bc75-19164b9ce845 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:36.649129 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-d41d4ddb-a513-4925-bc75-19164b9ce845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:36.664985 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d4ddb-a513-4925-bc75-19164b9ce845 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:36.669664 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:36.670304 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:36.670580 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:36.670850 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] PATH_INFO: `/v3/projects/1209c934a04845cabc02dd4236f28609` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:36.672642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1209c934a04845cabc02dd4236f28609)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:36.675027 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:36.714706 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:36.723274 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1209c934a04845cabc02dd4236f28609', 'request_id': 'req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.729655 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1209c934a04845cabc02dd4236f28609', 'request_id': 'req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.735748 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1209c934a04845cabc02dd4236f28609', 'request_id': 'req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.747773 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] The token cache is being invalidate because project 1209c934a04845cabc02dd4236f28609 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:36.748061 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.748674 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.750177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1449/2880] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:36 2022] DELETE /identity/v3/projects/1209c934a04845cabc02dd4236f28609 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:36.754475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027b120-af93-466e-8440-8382b2ab2ac2 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:36.764143 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9027b120-af93-466e-8440-8382b2ab2ac2 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:36.820747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c0ca2b-03bd-45e4-93d8-f4ba8b07880a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:36.821396 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0ca2b-03bd-45e4-93d8-f4ba8b07880a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:36.821710 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0ca2b-03bd-45e4-93d8-f4ba8b07880a admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:36.821963 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c0ca2b-03bd-45e4-93d8-f4ba8b07880a admin admin] PATH_INFO: `/v3/users/dbd2e1a4d8df422c85d8a3b90867a9fe` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:36.823072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0ca2b-03bd-45e4-93d8-f4ba8b07880a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbd2e1a4d8df422c85d8a3b90867a9fe)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:36.825011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c0ca2b-03bd-45e4-93d8-f4ba8b07880a admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:36.826431 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:36.872074 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-09c0ca2b-03bd-45e4-93d8-f4ba8b07880a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbd2e1a4d8df422c85d8a3b90867a9fe'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.872747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-09c0ca2b-03bd-45e4-93d8-f4ba8b07880a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbd2e1a4d8df422c85d8a3b90867a9fe'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.879824 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-09c0ca2b-03bd-45e4-93d8-f4ba8b07880a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbd2e1a4d8df422c85d8a3b90867a9fe'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.880459 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-09c0ca2b-03bd-45e4-93d8-f4ba8b07880a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbd2e1a4d8df422c85d8a3b90867a9fe'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.892681 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-09c0ca2b-03bd-45e4-93d8-f4ba8b07880a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbd2e1a4d8df422c85d8a3b90867a9fe'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:36.905421 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1432/2881] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:36 2022] DELETE /identity/v3/users/dbd2e1a4d8df422c85d8a3b90867a9fe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:37.178896 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:37.180324 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:37.194868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7ffb42-ea08-4d88-8c86-fa7607ef8b12 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:37.252083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:37.252752 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:37.252973 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:37.253276 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] PATH_INFO: `/v3/projects/08f942bc644640b6ba566989ffec1114` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:37.254670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f942bc644640b6ba566989ffec1114)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:37.256536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:37.304994 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f942bc644640b6ba566989ffec1114', 'request_id': 'req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:37.310940 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f942bc644640b6ba566989ffec1114', 'request_id': 'req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:37.316678 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f942bc644640b6ba566989ffec1114', 'request_id': 'req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:37.328877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] The token cache is being invalidate because project 08f942bc644640b6ba566989ffec1114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:37.329144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:37.329765 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-8e8b34f9-cdcd-40d0-be3f-ce7ee8f5f895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:37.331251 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1450/2882] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:37 2022] DELETE /identity/v3/projects/08f942bc644640b6ba566989ffec1114 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:37.336282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c8072-2014-4719-9ab3-4f895c3db65d None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:37.336751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c8072-2014-4719-9ab3-4f895c3db65d None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:37.336873 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c8072-2014-4719-9ab3-4f895c3db65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:37.356595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-625c8072-2014-4719-9ab3-4f895c3db65d None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:37.357832 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-625c8072-2014-4719-9ab3-4f895c3db65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:37.417309 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1433/2883] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:37.425623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257469d-3d58-4d5e-b1c2-b5acbf8e3f40 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:37.425911 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257469d-3d58-4d5e-b1c2-b5acbf8e3f40 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:37.426188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6257469d-3d58-4d5e-b1c2-b5acbf8e3f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:37.445670 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-6257469d-3d58-4d5e-b1c2-b5acbf8e3f40 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:37.446868 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-6257469d-3d58-4d5e-b1c2-b5acbf8e3f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:37.455547 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1451/2884] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:37.460004 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c8072-2014-4719-9ab3-4f895c3db65d None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:37.470302 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625c8072-2014-4719-9ab3-4f895c3db65d None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:37.474899 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680b2e3-a81e-4d1e-87e9-6182b95914f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:37.475490 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680b2e3-a81e-4d1e-87e9-6182b95914f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:37.475785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680b2e3-a81e-4d1e-87e9-6182b95914f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:37.476034 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b680b2e3-a81e-4d1e-87e9-6182b95914f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:37.476668 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680b2e3-a81e-4d1e-87e9-6182b95914f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81628) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:26:37.476968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680b2e3-a81e-4d1e-87e9-6182b95914f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:37.478587 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b680b2e3-a81e-4d1e-87e9-6182b95914f2 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:37.490561 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1434/2885] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 22 22:26:37.495229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257469d-3d58-4d5e-b1c2-b5acbf8e3f40 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:37.504630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6257469d-3d58-4d5e-b1c2-b5acbf8e3f40 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:37.509325 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb3ad8-e17f-4f16-8a4f-e6d88c397dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:37.509913 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3ad8-e17f-4f16-8a4f-e6d88c397dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:37.510188 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3ad8-e17f-4f16-8a4f-e6d88c397dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:37.510455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb3ad8-e17f-4f16-8a4f-e6d88c397dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:37.511087 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb3ad8-e17f-4f16-8a4f-e6d88c397dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:37.512958 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fb3ad8-e17f-4f16-8a4f-e6d88c397dc8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:37.540036 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1452/2886] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:37 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:37.544272 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680b2e3-a81e-4d1e-87e9-6182b95914f2 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:37.554230 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b680b2e3-a81e-4d1e-87e9-6182b95914f2 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:37.610658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3365be-1c77-436e-9213-37729f20e602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:37.611178 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3365be-1c77-436e-9213-37729f20e602 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:37.611456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3365be-1c77-436e-9213-37729f20e602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:37.611726 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3365be-1c77-436e-9213-37729f20e602 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:37.612356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3365be-1c77-436e-9213-37729f20e602 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:37.614297 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3365be-1c77-436e-9213-37729f20e602 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:37.625080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:37.639830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1435/2887] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:37 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:37.641634 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb3ad8-e17f-4f16-8a4f-e6d88c397dc8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:37.651891 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb3ad8-e17f-4f16-8a4f-e6d88c397dc8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:37.657340 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ca8a4-8e20-4315-b8d1-68809ab1bd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:37.658045 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ca8a4-8e20-4315-b8d1-68809ab1bd15 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:37.658344 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ca8a4-8e20-4315-b8d1-68809ab1bd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:37.658625 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25ca8a4-8e20-4315-b8d1-68809ab1bd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:37.659255 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ca8a4-8e20-4315-b8d1-68809ab1bd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:37.660838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25ca8a4-8e20-4315-b8d1-68809ab1bd15 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:37.670861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1453/2888] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:26:37.675160 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3365be-1c77-436e-9213-37729f20e602 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:37.684999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3365be-1c77-436e-9213-37729f20e602 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:37.689767 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c8b16-a690-4194-aa3b-3f80dc7d3fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:37.690353 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c8b16-a690-4194-aa3b-3f80dc7d3fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:37.690626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c8b16-a690-4194-aa3b-3f80dc7d3fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:37.690894 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c8b16-a690-4194-aa3b-3f80dc7d3fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:37.691522 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c8b16-a690-4194-aa3b-3f80dc7d3fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:37.693188 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708c8b16-a690-4194-aa3b-3f80dc7d3fd8 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:37.704064 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1436/2889] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:37.708171 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ca8a4-8e20-4315-b8d1-68809ab1bd15 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:37.718366 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25ca8a4-8e20-4315-b8d1-68809ab1bd15 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:37.723751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f891f6-f5de-4b9b-a165-1963a4d33896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:37.724405 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f891f6-f5de-4b9b-a165-1963a4d33896 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:37.724681 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f891f6-f5de-4b9b-a165-1963a4d33896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:37.724954 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f891f6-f5de-4b9b-a165-1963a4d33896 admin admin] PATH_INFO: `/v3/projects/ba23f952c4314842a4c27645c7379a34/users/7eeb121c68c74b66af3ab1fe545b87df/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:37.739280 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f891f6-f5de-4b9b-a165-1963a4d33896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba23f952c4314842a4c27645c7379a34, user_id=7eeb121c68c74b66af3ab1fe545b87df, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:37.741934 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f891f6-f5de-4b9b-a165-1963a4d33896 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:37.744091 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:37.755791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1454/2890] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:37 2022] PUT /identity/v3/projects/ba23f952c4314842a4c27645c7379a34/users/7eeb121c68c74b66af3ab1fe545b87df/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:37.767808 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6671a9-cc6c-413c-b1bd-cbd3c5c4428a None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:37.768089 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6671a9-cc6c-413c-b1bd-cbd3c5c4428a None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:37.768387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6671a9-cc6c-413c-b1bd-cbd3c5c4428a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:37.795950 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-cb6671a9-cc6c-413c-b1bd-cbd3c5c4428a None None] MFA Rules not processed for user `7eeb121c68c74b66af3ab1fe545b87df`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:37.797364 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-cb6671a9-cc6c-413c-b1bd-cbd3c5c4428a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:37.861721 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1437/2891] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:37.866529 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f891f6-f5de-4b9b-a165-1963a4d33896 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:37.928623 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f891f6-f5de-4b9b-a165-1963a4d33896 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:37.933192 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb86c2-3b47-4bc4-b2cd-e636a37fe2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:37.933787 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb86c2-3b47-4bc4-b2cd-e636a37fe2ff admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:37.934062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb86c2-3b47-4bc4-b2cd-e636a37fe2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:37.934329 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb86c2-3b47-4bc4-b2cd-e636a37fe2ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:37.934955 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb86c2-3b47-4bc4-b2cd-e636a37fe2ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:37.936858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcb86c2-3b47-4bc4-b2cd-e636a37fe2ff admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:37.964861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1455/2892] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:37 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:37.969035 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6671a9-cc6c-413c-b1bd-cbd3c5c4428a None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:37.979669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6671a9-cc6c-413c-b1bd-cbd3c5c4428a None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:38.041347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3861e96-0c47-4013-9ad7-58891b19a31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:38.041947 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861e96-0c47-4013-9ad7-58891b19a31f admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.042222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861e96-0c47-4013-9ad7-58891b19a31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.042489 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861e96-0c47-4013-9ad7-58891b19a31f admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.043119 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3861e96-0c47-4013-9ad7-58891b19a31f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:38.044999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3861e96-0c47-4013-9ad7-58891b19a31f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:38.067219 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1438/2893] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:37 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:26:38.071205 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb86c2-3b47-4bc4-b2cd-e636a37fe2ff admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:38.082073 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb86c2-3b47-4bc4-b2cd-e636a37fe2ff admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:38.086856 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3899a-89e1-458b-9e6b-820eb42d1453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:38.087438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3899a-89e1-458b-9e6b-820eb42d1453 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.087712 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3899a-89e1-458b-9e6b-820eb42d1453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.087980 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3899a-89e1-458b-9e6b-820eb42d1453 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.088640 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3899a-89e1-458b-9e6b-820eb42d1453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:38.090249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3899a-89e1-458b-9e6b-820eb42d1453 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:38.100546 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1456/2894] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 22 22:26:38.104231 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3861e96-0c47-4013-9ad7-58891b19a31f admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:38.114280 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3861e96-0c47-4013-9ad7-58891b19a31f admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:38.118773 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5acd9-c384-49d5-a997-a209e58d8e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:38.119355 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5acd9-c384-49d5-a997-a209e58d8e9f admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.119632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5acd9-c384-49d5-a997-a209e58d8e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.119900 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5acd9-c384-49d5-a997-a209e58d8e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.120532 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5acd9-c384-49d5-a997-a209e58d8e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:38.122177 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5acd9-c384-49d5-a997-a209e58d8e9f admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:38.132473 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1439/2895] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:38.136481 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3899a-89e1-458b-9e6b-820eb42d1453 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:38.147000 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3899a-89e1-458b-9e6b-820eb42d1453 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:38.151646 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7b8d3-5d1d-4432-b34c-0cfcba6f66c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:38.152286 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7b8d3-5d1d-4432-b34c-0cfcba6f66c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.152572 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7b8d3-5d1d-4432-b34c-0cfcba6f66c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.152846 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7b8d3-5d1d-4432-b34c-0cfcba6f66c2 admin admin] PATH_INFO: `/v3/projects/f81c605ff1d843f3b307f7d0d68664e5/users/6ff15f2fb1d44863b20c70b7574bf6f9/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.167621 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7b8d3-5d1d-4432-b34c-0cfcba6f66c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f81c605ff1d843f3b307f7d0d68664e5, user_id=6ff15f2fb1d44863b20c70b7574bf6f9, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:38.170281 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7b8d3-5d1d-4432-b34c-0cfcba6f66c2 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:38.183450 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1457/2896] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:38 2022] PUT /identity/v3/projects/f81c605ff1d843f3b307f7d0d68664e5/users/6ff15f2fb1d44863b20c70b7574bf6f9/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:38.193646 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb40088-9efe-4bec-8794-3139083e4c6f None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.193957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb40088-9efe-4bec-8794-3139083e4c6f None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.194228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb40088-9efe-4bec-8794-3139083e4c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.222738 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-7eb40088-9efe-4bec-8794-3139083e4c6f None None] MFA Rules not processed for user `6ff15f2fb1d44863b20c70b7574bf6f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:38.223941 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7eb40088-9efe-4bec-8794-3139083e4c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:38.263890 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:38.290485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1440/2897] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:38.295508 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7b8d3-5d1d-4432-b34c-0cfcba6f66c2 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:38.358127 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7b8d3-5d1d-4432-b34c-0cfcba6f66c2 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:38.362753 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f2d31-bbdf-465b-a4aa-500e8b767e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:38.363348 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f2d31-bbdf-465b-a4aa-500e8b767e14 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.363620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f2d31-bbdf-465b-a4aa-500e8b767e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.363888 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f2d31-bbdf-465b-a4aa-500e8b767e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.364583 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f2d31-bbdf-465b-a4aa-500e8b767e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:38.366436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7f2d31-bbdf-465b-a4aa-500e8b767e14 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:38.392892 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1458/2898] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:38 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:38.396960 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb40088-9efe-4bec-8794-3139083e4c6f None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:38.408068 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb40088-9efe-4bec-8794-3139083e4c6f None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:38.463011 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685739b4-a598-497e-ae01-550de6733869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:38.463601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685739b4-a598-497e-ae01-550de6733869 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.463882 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685739b4-a598-497e-ae01-550de6733869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.464149 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685739b4-a598-497e-ae01-550de6733869 admin admin] PATH_INFO: `/v3/users` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.464779 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685739b4-a598-497e-ae01-550de6733869 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:38.466707 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685739b4-a598-497e-ae01-550de6733869 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:38.487451 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1441/2899] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:38 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:38.491418 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f2d31-bbdf-465b-a4aa-500e8b767e14 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:38.501536 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f2d31-bbdf-465b-a4aa-500e8b767e14 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:38.506086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e6bcc-1bf5-49d9-a8de-b67a3ac6cfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:38.506671 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e6bcc-1bf5-49d9-a8de-b67a3ac6cfc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.506946 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e6bcc-1bf5-49d9-a8de-b67a3ac6cfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.507211 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e6bcc-1bf5-49d9-a8de-b67a3ac6cfc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.507842 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e6bcc-1bf5-49d9-a8de-b67a3ac6cfc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:38.509430 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e6bcc-1bf5-49d9-a8de-b67a3ac6cfc8 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:38.520263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1459/2900] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:38.524070 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685739b4-a598-497e-ae01-550de6733869 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:38.534453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685739b4-a598-497e-ae01-550de6733869 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:38.539048 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffc191-b883-4e48-add0-cd995b0656cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:38.539680 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffc191-b883-4e48-add0-cd995b0656cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.539957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffc191-b883-4e48-add0-cd995b0656cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.540229 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffc191-b883-4e48-add0-cd995b0656cf admin admin] PATH_INFO: `/v3/projects/8d9065ab2ed343a8b863fc38ee1f4550/users/e75eea694fe84557bf6d0cb0025ab034/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.556176 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ffc191-b883-4e48-add0-cd995b0656cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9065ab2ed343a8b863fc38ee1f4550, user_id=e75eea694fe84557bf6d0cb0025ab034, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:38.559218 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ffc191-b883-4e48-add0-cd995b0656cf admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:38.572169 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1442/2901] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:38 2022] PUT /identity/v3/projects/8d9065ab2ed343a8b863fc38ee1f4550/users/e75eea694fe84557bf6d0cb0025ab034/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:38.576201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e6bcc-1bf5-49d9-a8de-b67a3ac6cfc8 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:38.587123 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e6bcc-1bf5-49d9-a8de-b67a3ac6cfc8 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:38.599493 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:38.644248 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c59ce-dd61-4894-b423-4bde0a63f6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:38.644781 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c59ce-dd61-4894-b423-4bde0a63f6dc admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.645063 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c59ce-dd61-4894-b423-4bde0a63f6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.645371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c59ce-dd61-4894-b423-4bde0a63f6dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.646006 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c59ce-dd61-4894-b423-4bde0a63f6dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:38.647609 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c59ce-dd61-4894-b423-4bde0a63f6dc admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:38.657750 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1460/2902] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:38.661957 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffc191-b883-4e48-add0-cd995b0656cf admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:38.672366 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffc191-b883-4e48-add0-cd995b0656cf admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:38.677806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fcc0d-b857-4759-8b51-205d2b8b2439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:38.678439 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fcc0d-b857-4759-8b51-205d2b8b2439 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.678762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fcc0d-b857-4759-8b51-205d2b8b2439 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.679015 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fcc0d-b857-4759-8b51-205d2b8b2439 admin admin] PATH_INFO: `/v3/projects/8d9065ab2ed343a8b863fc38ee1f4550/users/e75eea694fe84557bf6d0cb0025ab034/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.680900 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fcc0d-b857-4759-8b51-205d2b8b2439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d9065ab2ed343a8b863fc38ee1f4550, user_id=e75eea694fe84557bf6d0cb0025ab034, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:38.683559 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fcc0d-b857-4759-8b51-205d2b8b2439 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:38.693706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1443/2903] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:38 2022] PUT /identity/v3/projects/8d9065ab2ed343a8b863fc38ee1f4550/users/e75eea694fe84557bf6d0cb0025ab034/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:38.704051 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0915d2-e479-4be0-80cb-749b2160364b None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.704364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0915d2-e479-4be0-80cb-749b2160364b None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.704636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0915d2-e479-4be0-80cb-749b2160364b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.733789 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-1d0915d2-e479-4be0-80cb-749b2160364b None None] MFA Rules not processed for user `e75eea694fe84557bf6d0cb0025ab034`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:38.735897 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1d0915d2-e479-4be0-80cb-749b2160364b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:38.802593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1461/2904] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:38.813708 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fcc0d-b857-4759-8b51-205d2b8b2439 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:38.815395 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d78fcc0d-b857-4759-8b51-205d2b8b2439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:38.873296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fcc0d-b857-4759-8b51-205d2b8b2439 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:38.877088 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481d87d-958b-470a-a569-83bece3e9458 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:38.877706 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481d87d-958b-470a-a569-83bece3e9458 service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:38.877980 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481d87d-958b-470a-a569-83bece3e9458 service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:38.878247 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0481d87d-958b-470a-a569-83bece3e9458 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:38.924725 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:38.935298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481d87d-958b-470a-a569-83bece3e9458 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:38.936751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0481d87d-958b-470a-a569-83bece3e9458 service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:38.945606 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1444/2905] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:26:40.445885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0915d2-e479-4be0-80cb-749b2160364b None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:40.447349 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-1d0915d2-e479-4be0-80cb-749b2160364b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:40.460078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0915d2-e479-4be0-80cb-749b2160364b None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:40.463659 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfb1cd-c9de-4c9b-b8bb-c7489ad15a8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:40.464253 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfb1cd-c9de-4c9b-b8bb-c7489ad15a8e service cinder] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:40.464531 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfb1cd-c9de-4c9b-b8bb-c7489ad15a8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:40.464801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfb1cd-c9de-4c9b-b8bb-c7489ad15a8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:40.520311 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfb1cd-c9de-4c9b-b8bb-c7489ad15a8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:40.521902 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfb1cd-c9de-4c9b-b8bb-c7489ad15a8e service cinder] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:40.530743 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1462/2906] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:41.063687 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481d87d-958b-470a-a569-83bece3e9458 service cinder] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:41.072403 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0481d87d-958b-470a-a569-83bece3e9458 service cinder] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:41.076026 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dfb303-2560-4a1d-ba5c-d2ed12e7b35a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcfdbb6b3dd04478ae99681cc649fdb9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:41.076615 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfb303-2560-4a1d-ba5c-d2ed12e7b35a service cinder] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:41.076892 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfb303-2560-4a1d-ba5c-d2ed12e7b35a service cinder] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:41.077205 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dfb303-2560-4a1d-ba5c-d2ed12e7b35a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:41.086375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfb303-2560-4a1d-ba5c-d2ed12e7b35a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:41.087783 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dfb303-2560-4a1d-ba5c-d2ed12e7b35a service cinder] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:41.097823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1445/2907] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:44.967811 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfb1cd-c9de-4c9b-b8bb-c7489ad15a8e service cinder] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:45.008314 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=100 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:45.030586 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfb1cd-c9de-4c9b-b8bb-c7489ad15a8e service cinder] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:45.036497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:45.037168 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:45.037444 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:45.037713 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] PATH_INFO: `/v3/users/7eeb121c68c74b66af3ab1fe545b87df` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:45.038818 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eeb121c68c74b66af3ab1fe545b87df)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:45.040699 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:45.089149 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eeb121c68c74b66af3ab1fe545b87df'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.089799 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eeb121c68c74b66af3ab1fe545b87df'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.090402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eeb121c68c74b66af3ab1fe545b87df'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.102661 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eeb121c68c74b66af3ab1fe545b87df'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.109380 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eeb121c68c74b66af3ab1fe545b87df'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.121827 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1463/2908] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:44 2022] DELETE /identity/v3/users/7eeb121c68c74b66af3ab1fe545b87df => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:45.126427 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62dbfa-a045-4285-85f7-8a1c5282a5d4 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:45.126935 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62dbfa-a045-4285-85f7-8a1c5282a5d4 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:45.127053 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62dbfa-a045-4285-85f7-8a1c5282a5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:45.145825 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6e62dbfa-a045-4285-85f7-8a1c5282a5d4 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:45.147001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6e62dbfa-a045-4285-85f7-8a1c5282a5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:45.204253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1446/2909] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:45.214135 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:45.215518 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:45.271383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9724e7-3e6e-4d6e-bf67-bf9915da2b59 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:45.274984 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efffee-9c85-4127-9f5b-5fdf27d696ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:45.275582 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efffee-9c85-4127-9f5b-5fdf27d696ec service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:45.275861 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efffee-9c85-4127-9f5b-5fdf27d696ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:45.276131 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efffee-9c85-4127-9f5b-5fdf27d696ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:45.284876 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efffee-9c85-4127-9f5b-5fdf27d696ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:45.286321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efffee-9c85-4127-9f5b-5fdf27d696ec service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:45.295464 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1464/2910] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:45.558617 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62dbfa-a045-4285-85f7-8a1c5282a5d4 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:45.560296 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6e62dbfa-a045-4285-85f7-8a1c5282a5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:45.573585 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62dbfa-a045-4285-85f7-8a1c5282a5d4 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:45.578771 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:45.579653 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:45.579931 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:45.580201 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] PATH_INFO: `/v3/projects/ba23f952c4314842a4c27645c7379a34` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:45.582211 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba23f952c4314842a4c27645c7379a34)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:45.585075 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:45.630991 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba23f952c4314842a4c27645c7379a34', 'request_id': 'req-31568cfa-c1cc-4c2e-9544-61ca1bea325c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.638301 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba23f952c4314842a4c27645c7379a34', 'request_id': 'req-31568cfa-c1cc-4c2e-9544-61ca1bea325c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.644734 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba23f952c4314842a4c27645c7379a34', 'request_id': 'req-31568cfa-c1cc-4c2e-9544-61ca1bea325c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.655563 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] The token cache is being invalidate because project ba23f952c4314842a4c27645c7379a34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:45.655850 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.657014 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.658626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1447/2911] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:45 2022] DELETE /identity/v3/projects/ba23f952c4314842a4c27645c7379a34 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:45.662720 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efffee-9c85-4127-9f5b-5fdf27d696ec service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:45.672932 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efffee-9c85-4127-9f5b-5fdf27d696ec service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:45.728552 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:45.729201 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:45.729479 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:45.729744 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] PATH_INFO: `/v3/users/6ff15f2fb1d44863b20c70b7574bf6f9` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:45.730845 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff15f2fb1d44863b20c70b7574bf6f9)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:45.732722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:45.734177 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:45.780559 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff15f2fb1d44863b20c70b7574bf6f9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.781226 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff15f2fb1d44863b20c70b7574bf6f9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.782691 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff15f2fb1d44863b20c70b7574bf6f9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.794396 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff15f2fb1d44863b20c70b7574bf6f9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.801282 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff15f2fb1d44863b20c70b7574bf6f9'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:45.813620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1465/2912] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:45 2022] DELETE /identity/v3/users/6ff15f2fb1d44863b20c70b7574bf6f9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:46.086320 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:46.087750 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:46.100932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31568cfa-c1cc-4c2e-9544-61ca1bea325c admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:46.158842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:46.159529 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:46.159728 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:46.159999 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] PATH_INFO: `/v3/projects/f81c605ff1d843f3b307f7d0d68664e5` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:46.161126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f81c605ff1d843f3b307f7d0d68664e5)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:46.163029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:46.164413 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:46.210884 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f81c605ff1d843f3b307f7d0d68664e5', 'request_id': 'req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.218494 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f81c605ff1d843f3b307f7d0d68664e5', 'request_id': 'req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.225080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f81c605ff1d843f3b307f7d0d68664e5', 'request_id': 'req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.235844 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] The token cache is being invalidate because project f81c605ff1d843f3b307f7d0d68664e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:46.236148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.236806 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.238276 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1448/2913] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:46 2022] DELETE /identity/v3/projects/f81c605ff1d843f3b307f7d0d68664e5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:46.242215 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:46.243617 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:46.256323 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57fce07-53fc-43e0-a4c0-50122111fbfb admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:46.314560 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cbb9e0-d1db-40de-917e-a08e9204c04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:46.315187 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbb9e0-d1db-40de-917e-a08e9204c04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:46.315461 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbb9e0-d1db-40de-917e-a08e9204c04d admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:46.315731 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cbb9e0-d1db-40de-917e-a08e9204c04d admin admin] PATH_INFO: `/v3/users/e75eea694fe84557bf6d0cb0025ab034` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:46.316841 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbb9e0-d1db-40de-917e-a08e9204c04d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e75eea694fe84557bf6d0cb0025ab034)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:46.318838 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cbb9e0-d1db-40de-917e-a08e9204c04d admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:46.365620 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-06cbb9e0-d1db-40de-917e-a08e9204c04d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e75eea694fe84557bf6d0cb0025ab034'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.366198 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-06cbb9e0-d1db-40de-917e-a08e9204c04d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e75eea694fe84557bf6d0cb0025ab034'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.366797 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-06cbb9e0-d1db-40de-917e-a08e9204c04d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e75eea694fe84557bf6d0cb0025ab034'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.379319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-06cbb9e0-d1db-40de-917e-a08e9204c04d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e75eea694fe84557bf6d0cb0025ab034'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.386078 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-06cbb9e0-d1db-40de-917e-a08e9204c04d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e75eea694fe84557bf6d0cb0025ab034'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.394184 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:46.400209 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1466/2914] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:26:46 2022] DELETE /identity/v3/users/e75eea694fe84557bf6d0cb0025ab034 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:46.673425 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:46.674878 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:46.688257 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4b9e8-b6f0-457e-a990-2baa9ae5855e admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:46.748063 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:46.748683 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:46.748958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:46.749260 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] PATH_INFO: `/v3/projects/8d9065ab2ed343a8b863fc38ee1f4550` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:46.750442 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d9065ab2ed343a8b863fc38ee1f4550)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:46.752332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:46.797866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d9065ab2ed343a8b863fc38ee1f4550', 'request_id': 'req-9c665c28-3c64-4dcb-9423-99503e21694c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.805837 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d9065ab2ed343a8b863fc38ee1f4550', 'request_id': 'req-9c665c28-3c64-4dcb-9423-99503e21694c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.811862 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d9065ab2ed343a8b863fc38ee1f4550', 'request_id': 'req-9c665c28-3c64-4dcb-9423-99503e21694c'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.822715 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] The token cache is being invalidate because project 8d9065ab2ed343a8b863fc38ee1f4550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81628) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:26:46.823001 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.823626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-9c665c28-3c64-4dcb-9423-99503e21694c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:26:46.825073 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1449/2915] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:26:46 2022] DELETE /identity/v3/projects/8d9065ab2ed343a8b863fc38ee1f4550 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:46.830883 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92824c3f-1f4d-41f1-a3c1-c27ded285750 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:46.831174 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92824c3f-1f4d-41f1-a3c1-c27ded285750 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:46.831455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92824c3f-1f4d-41f1-a3c1-c27ded285750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:46.851384 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-92824c3f-1f4d-41f1-a3c1-c27ded285750 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:46.852593 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-92824c3f-1f4d-41f1-a3c1-c27ded285750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:46.913783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1467/2916] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 22 22:26:46.922689 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7cf5e-268b-46bd-983e-b2ce169c0819 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:46.922951 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7cf5e-268b-46bd-983e-b2ce169c0819 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:46.923222 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7cf5e-268b-46bd-983e-b2ce169c0819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:46.944282 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-c5b7cf5e-268b-46bd-983e-b2ce169c0819 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:46.945657 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c5b7cf5e-268b-46bd-983e-b2ce169c0819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:46.954319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1450/2917] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:46.958754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92824c3f-1f4d-41f1-a3c1-c27ded285750 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:46.968763 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92824c3f-1f4d-41f1-a3c1-c27ded285750 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:46.973929 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc776f-7b8e-4e50-a180-6c860844a3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:46.974527 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc776f-7b8e-4e50-a180-6c860844a3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:46.974809 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc776f-7b8e-4e50-a180-6c860844a3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:46.975083 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc776f-7b8e-4e50-a180-6c860844a3f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:46.975756 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc776f-7b8e-4e50-a180-6c860844a3f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81629) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 22 22:26:46.976062 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc776f-7b8e-4e50-a180-6c860844a3f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:46.977680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc776f-7b8e-4e50-a180-6c860844a3f7 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:46.989436 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1468/2918] 10.222.0.25 () {58 vars in 1153 bytes} [Thu Sep 22 22:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 22 22:26:46.993595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7cf5e-268b-46bd-983e-b2ce169c0819 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:47.003209 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7cf5e-268b-46bd-983e-b2ce169c0819 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:47.007886 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae4e1e-bf9c-473b-bb92-6c15080980b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:47.008475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae4e1e-bf9c-473b-bb92-6c15080980b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:47.008755 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae4e1e-bf9c-473b-bb92-6c15080980b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:47.009029 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae4e1e-bf9c-473b-bb92-6c15080980b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:47.009705 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae4e1e-bf9c-473b-bb92-6c15080980b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:47.011600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae4e1e-bf9c-473b-bb92-6c15080980b0 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:47.023429 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:47.041298 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1451/2919] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:46 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:47.045339 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc776f-7b8e-4e50-a180-6c860844a3f7 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:47.057325 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc776f-7b8e-4e50-a180-6c860844a3f7 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:47.114663 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cedfcb-1be6-41e9-8f5e-2fe2edc5d5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:47.115259 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cedfcb-1be6-41e9-8f5e-2fe2edc5d5e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:47.115532 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cedfcb-1be6-41e9-8f5e-2fe2edc5d5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:47.115801 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cedfcb-1be6-41e9-8f5e-2fe2edc5d5e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:47.116421 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cedfcb-1be6-41e9-8f5e-2fe2edc5d5e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:47.118330 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cedfcb-1be6-41e9-8f5e-2fe2edc5d5e9 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:47.140272 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1469/2920] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:47 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:47.145126 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae4e1e-bf9c-473b-bb92-6c15080980b0 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:47.155314 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae4e1e-bf9c-473b-bb92-6c15080980b0 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:47.160581 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedaa23-69dd-4362-8707-8c41386085ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:47.161207 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedaa23-69dd-4362-8707-8c41386085ee admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:47.161463 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedaa23-69dd-4362-8707-8c41386085ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:47.161729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedaa23-69dd-4362-8707-8c41386085ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:47.162391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedaa23-69dd-4362-8707-8c41386085ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:47.164030 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedaa23-69dd-4362-8707-8c41386085ee admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:47.174829 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1452/2921] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:47.178874 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cedfcb-1be6-41e9-8f5e-2fe2edc5d5e9 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:47.188777 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cedfcb-1be6-41e9-8f5e-2fe2edc5d5e9 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:47.193336 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25e489-e66d-4d69-aea1-dfa6806974b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:47.193920 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25e489-e66d-4d69-aea1-dfa6806974b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:47.194194 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25e489-e66d-4d69-aea1-dfa6806974b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:47.194466 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25e489-e66d-4d69-aea1-dfa6806974b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:47.195100 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25e489-e66d-4d69-aea1-dfa6806974b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:47.196666 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25e489-e66d-4d69-aea1-dfa6806974b3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:47.207700 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1470/2922] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:47.211929 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedaa23-69dd-4362-8707-8c41386085ee admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:47.222722 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedaa23-69dd-4362-8707-8c41386085ee admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:47.227293 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05924a8-89de-490d-bd3f-54f847b43b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:47.227922 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05924a8-89de-490d-bd3f-54f847b43b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:47.228197 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05924a8-89de-490d-bd3f-54f847b43b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:47.228465 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05924a8-89de-490d-bd3f-54f847b43b21 admin admin] PATH_INFO: `/v3/projects/285dd074069d48d2951a195965088521/users/27bbd818457b4a20823fe0ec0601c312/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:47.242865 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05924a8-89de-490d-bd3f-54f847b43b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285dd074069d48d2951a195965088521, user_id=27bbd818457b4a20823fe0ec0601c312, role_id=d153f6e7d5e14dd7ba4f1de3f35cc0be)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:47.245513 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05924a8-89de-490d-bd3f-54f847b43b21 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:47.257921 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1453/2923] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:47 2022] PUT /identity/v3/projects/285dd074069d48d2951a195965088521/users/27bbd818457b4a20823fe0ec0601c312/roles/d153f6e7d5e14dd7ba4f1de3f35cc0be => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:47.263013 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d8f8-a691-46c9-8230-e98253671cd0 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:47.263497 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d8f8-a691-46c9-8230-e98253671cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:47.263622 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee40d8f8-a691-46c9-8230-e98253671cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:47.283354 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-ee40d8f8-a691-46c9-8230-e98253671cd0 None None] MFA Rules not processed for user `9a8dd819f6d742ceba5a020c855827f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:47.284541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ee40d8f8-a691-46c9-8230-e98253671cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:47.341228 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1471/2924] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:47.351389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05924a8-89de-490d-bd3f-54f847b43b21 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:47.353072 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-d05924a8-89de-490d-bd3f-54f847b43b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:47.412785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05924a8-89de-490d-bd3f-54f847b43b21 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:47.416519 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e1699-d99d-4a5d-9066-2cdcb939a7ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:47.417056 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e1699-d99d-4a5d-9066-2cdcb939a7ad service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:47.417375 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e1699-d99d-4a5d-9066-2cdcb939a7ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:47.417658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09e1699-d99d-4a5d-9066-2cdcb939a7ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:47.426616 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e1699-d99d-4a5d-9066-2cdcb939a7ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:47.428037 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09e1699-d99d-4a5d-9066-2cdcb939a7ad service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:47.437356 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1454/2925] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:51.375837 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514952b8-50d1-429d-a734-79327f5b7bb7 None None] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:51.376383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514952b8-50d1-429d-a734-79327f5b7bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:51.376383 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514952b8-50d1-429d-a734-79327f5b7bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:51.384719 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:51.412256 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.auth.core [None req-514952b8-50d1-429d-a734-79327f5b7bb7 None None] MFA Rules not processed for user `27bbd818457b4a20823fe0ec0601c312`. Rule list: `[]` (Enabled: `True`). {{(pid=81629) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:51.413570 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-514952b8-50d1-429d-a734-79327f5b7bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:51.483364 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1472/2926] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:51.488846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e1699-d99d-4a5d-9066-2cdcb939a7ad service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:51.498701 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09e1699-d99d-4a5d-9066-2cdcb939a7ad service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:51.503714 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80a545-5093-4a07-ba79-ba55b7a1e940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:51.504323 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80a545-5093-4a07-ba79-ba55b7a1e940 admin admin] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:51.504600 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80a545-5093-4a07-ba79-ba55b7a1e940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:51.504869 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80a545-5093-4a07-ba79-ba55b7a1e940 admin admin] PATH_INFO: `/v3/projects` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:51.505551 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80a545-5093-4a07-ba79-ba55b7a1e940 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:51.507455 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d80a545-5093-4a07-ba79-ba55b7a1e940 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:51.536803 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1455/2927] 10.222.0.25 () {60 vars in 1153 bytes} [Thu Sep 22 22:26:51 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 22 22:26:51.540907 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514952b8-50d1-429d-a734-79327f5b7bb7 None None] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:51.552998 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514952b8-50d1-429d-a734-79327f5b7bb7 None None] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:51.614146 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e14942-3181-4a17-b9ec-50eac54cfabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:51.614692 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14942-3181-4a17-b9ec-50eac54cfabd admin admin] REQUEST_METHOD: `POST` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:51.614968 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14942-3181-4a17-b9ec-50eac54cfabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:51.615241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14942-3181-4a17-b9ec-50eac54cfabd admin admin] PATH_INFO: `/v3/users` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:51.615939 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e14942-3181-4a17-b9ec-50eac54cfabd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:51.617877 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e14942-3181-4a17-b9ec-50eac54cfabd admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:51.642263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1473/2928] 10.222.0.25 () {60 vars in 1144 bytes} [Thu Sep 22 22:26:51 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 22 22:26:51.646330 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80a545-5093-4a07-ba79-ba55b7a1e940 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:51.657863 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80a545-5093-4a07-ba79-ba55b7a1e940 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:51.662554 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9919b8a-7a16-4cba-aa8b-02f7c26e0858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:51.663059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9919b8a-7a16-4cba-aa8b-02f7c26e0858 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:51.663333 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9919b8a-7a16-4cba-aa8b-02f7c26e0858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:51.663601 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9919b8a-7a16-4cba-aa8b-02f7c26e0858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:51.664319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9919b8a-7a16-4cba-aa8b-02f7c26e0858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:51.665918 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9919b8a-7a16-4cba-aa8b-02f7c26e0858 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:51.678228 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1456/2929] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:51.682592 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e14942-3181-4a17-b9ec-50eac54cfabd admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:51.693371 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e14942-3181-4a17-b9ec-50eac54cfabd admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:51.698866 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4716bc-f994-4744-b078-5ba93b0277c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:51.699521 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4716bc-f994-4744-b078-5ba93b0277c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:51.699800 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4716bc-f994-4744-b078-5ba93b0277c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:51.700075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4716bc-f994-4744-b078-5ba93b0277c1 admin admin] PATH_INFO: `/v3/projects/1d561eb1ee914613a7e83b619c2bd885/users/f58d0f7397a34488a690a0352a2e9298/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:51.728432 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4716bc-f994-4744-b078-5ba93b0277c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d561eb1ee914613a7e83b619c2bd885, user_id=f58d0f7397a34488a690a0352a2e9298, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:51.731321 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4716bc-f994-4744-b078-5ba93b0277c1 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:51.745496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1474/2930] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:51 2022] PUT /identity/v3/projects/1d561eb1ee914613a7e83b619c2bd885/users/f58d0f7397a34488a690a0352a2e9298/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:51.749501 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9919b8a-7a16-4cba-aa8b-02f7c26e0858 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:51.764200 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9919b8a-7a16-4cba-aa8b-02f7c26e0858 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:51.823968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bee67e-eb63-4e8e-9d0c-f61416534e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:51.824580 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bee67e-eb63-4e8e-9d0c-f61416534e97 admin admin] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:51.824855 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bee67e-eb63-4e8e-9d0c-f61416534e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:51.825144 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bee67e-eb63-4e8e-9d0c-f61416534e97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:51.826079 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bee67e-eb63-4e8e-9d0c-f61416534e97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:51.827694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bee67e-eb63-4e8e-9d0c-f61416534e97 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:51.839101 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1457/2931] 10.222.0.25 () {58 vars in 1122 bytes} [Thu Sep 22 22:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 22 22:26:51.843277 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4716bc-f994-4744-b078-5ba93b0277c1 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:51.853931 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4716bc-f994-4744-b078-5ba93b0277c1 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:51.859019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612508c-5370-4d43-9001-455dd4260ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:51.860249 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612508c-5370-4d43-9001-455dd4260ac3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:51.860533 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612508c-5370-4d43-9001-455dd4260ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:51.860806 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612508c-5370-4d43-9001-455dd4260ac3 admin admin] PATH_INFO: `/v3/projects/1d561eb1ee914613a7e83b619c2bd885/users/f58d0f7397a34488a690a0352a2e9298/roles/c1595a2afa7947148e6cfe39b0e4990c` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:51.863150 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612508c-5370-4d43-9001-455dd4260ac3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d561eb1ee914613a7e83b619c2bd885, user_id=f58d0f7397a34488a690a0352a2e9298, role_id=c1595a2afa7947148e6cfe39b0e4990c)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:51.865735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612508c-5370-4d43-9001-455dd4260ac3 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:51.877680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1475/2932] 10.222.0.25 () {60 vars in 1483 bytes} [Thu Sep 22 22:26:51 2022] PUT /identity/v3/projects/1d561eb1ee914613a7e83b619c2bd885/users/f58d0f7397a34488a690a0352a2e9298/roles/c1595a2afa7947148e6cfe39b0e4990c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:26:56.241762 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ee9cb-d3e8-4c90-99d5-525570e349fb None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:56.243148 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ee9cb-d3e8-4c90-99d5-525570e349fb None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:56.243412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8ee9cb-d3e8-4c90-99d5-525570e349fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:56.283623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-6d8ee9cb-d3e8-4c90-99d5-525570e349fb None None] MFA Rules not processed for user `f58d0f7397a34488a690a0352a2e9298`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:56.284889 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6d8ee9cb-d3e8-4c90-99d5-525570e349fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:56.302868 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:56.388456 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1458/2933] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:56 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:26:56.404735 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612508c-5370-4d43-9001-455dd4260ac3 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:26:56.407970 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-5612508c-5370-4d43-9001-455dd4260ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:56.481449 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:26:56.514977 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612508c-5370-4d43-9001-455dd4260ac3 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:26:56.522401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a5518-cb65-463c-b6fb-a05483e41fd1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:26:56.523035 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a5518-cb65-463c-b6fb-a05483e41fd1 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:56.523305 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a5518-cb65-463c-b6fb-a05483e41fd1 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:56.523578 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a5518-cb65-463c-b6fb-a05483e41fd1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:56.592144 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a5518-cb65-463c-b6fb-a05483e41fd1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:26:56.593981 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099a5518-cb65-463c-b6fb-a05483e41fd1 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:26:56.603705 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1476/2934] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:26:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:26:58.607505 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ee700-3a10-469f-84d0-1a58916b0eb1 None None] REQUEST_METHOD: `POST` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:26:58.608016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ee700-3a10-469f-84d0-1a58916b0eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:26:58.608206 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ee700-3a10-469f-84d0-1a58916b0eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:26:58.628135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.auth.core [None req-069ee700-3a10-469f-84d0-1a58916b0eb1 None None] MFA Rules not processed for user `27bbd818457b4a20823fe0ec0601c312`. Rule list: `[]` (Enabled: `True`). {{(pid=81628) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 22 22:26:58.629410 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-069ee700-3a10-469f-84d0-1a58916b0eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:26:58.636163 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1459/2935] 10.222.0.25 () {58 vars in 958 bytes} [Thu Sep 22 22:26:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 22 22:27:06.599668 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=35 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:27:08.623174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=27 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:27:19.645819 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a5518-cb65-463c-b6fb-a05483e41fd1 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:27:19.647233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-099a5518-cb65-463c-b6fb-a05483e41fd1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:27:19.702386 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099a5518-cb65-463c-b6fb-a05483e41fd1 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:27:19.706667 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5de20-3919-4b1e-b5e1-224bd3b8ae93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:27:19.707263 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5de20-3919-4b1e-b5e1-224bd3b8ae93 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:27:19.707541 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5de20-3919-4b1e-b5e1-224bd3b8ae93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:27:19.707812 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd5de20-3919-4b1e-b5e1-224bd3b8ae93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:27:19.709642 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-ecd5de20-3919-4b1e-b5e1-224bd3b8ae93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:27:19.720075 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5de20-3919-4b1e-b5e1-224bd3b8ae93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:27:19.721619 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd5de20-3919-4b1e-b5e1-224bd3b8ae93 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:27:19.729986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1477/2936] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:27:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:27:20.179059 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ee700-3a10-469f-84d0-1a58916b0eb1 None None] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:27:20.180283 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-069ee700-3a10-469f-84d0-1a58916b0eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:27:20.253076 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ee700-3a10-469f-84d0-1a58916b0eb1 None None] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:27:20.258093 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1885e-bc82-42d7-a040-cd1630a916d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:27:20.258694 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1885e-bc82-42d7-a040-cd1630a916d2 service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:27:20.258974 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1885e-bc82-42d7-a040-cd1630a916d2 service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:27:20.259246 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b1885e-bc82-42d7-a040-cd1630a916d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:27:20.260983 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-27b1885e-bc82-42d7-a040-cd1630a916d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:27:20.272595 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1885e-bc82-42d7-a040-cd1630a916d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:27:20.274190 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b1885e-bc82-42d7-a040-cd1630a916d2 service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:27:20.285016 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1460/2937] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:27:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:27:22.172996 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5de20-3919-4b1e-b5e1-224bd3b8ae93 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:27:22.188019 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd5de20-3919-4b1e-b5e1-224bd3b8ae93 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:27:22.191885 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb6b5e-5846-4cb5-8d75-e09b7b157c66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:27:22.192484 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb6b5e-5846-4cb5-8d75-e09b7b157c66 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:27:22.192759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb6b5e-5846-4cb5-8d75-e09b7b157c66 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:27:22.193028 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb6b5e-5846-4cb5-8d75-e09b7b157c66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:27:22.194802 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-51fb6b5e-5846-4cb5-8d75-e09b7b157c66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:27:22.205401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb6b5e-5846-4cb5-8d75-e09b7b157c66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:27:22.206815 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb6b5e-5846-4cb5-8d75-e09b7b157c66 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:27:22.216455 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1478/2938] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:27:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:27:22.929632 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1885e-bc82-42d7-a040-cd1630a916d2 service placement] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:27:22.931135 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-27b1885e-bc82-42d7-a040-cd1630a916d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:27:22.949391 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b1885e-bc82-42d7-a040-cd1630a916d2 service placement] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:27:22.955660 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9988c-434f-42c8-adea-3e77c432f2a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:27:22.956253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9988c-434f-42c8-adea-3e77c432f2a3 service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:27:22.956533 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9988c-434f-42c8-adea-3e77c432f2a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:27:22.956797 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff9988c-434f-42c8-adea-3e77c432f2a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:27:22.958715 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-7ff9988c-434f-42c8-adea-3e77c432f2a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:27:22.975319 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9988c-434f-42c8-adea-3e77c432f2a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:27:22.977203 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff9988c-434f-42c8-adea-3e77c432f2a3 service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:27:22.988193 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1461/2939] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:27:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:27:32.211791 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=32 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:27:32.983412 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=38 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:27:35.583438 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb6b5e-5846-4cb5-8d75-e09b7b157c66 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:27:35.597453 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb6b5e-5846-4cb5-8d75-e09b7b157c66 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:27:35.602853 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d7c55-5be4-44e4-ba67-0d06a80b5292 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:27:35.603452 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7c55-5be4-44e4-ba67-0d06a80b5292 service nova] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:27:35.603725 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7c55-5be4-44e4-ba67-0d06a80b5292 service nova] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:27:35.603992 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040d7c55-5be4-44e4-ba67-0d06a80b5292 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:27:35.605722 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-040d7c55-5be4-44e4-ba67-0d06a80b5292 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:27:35.618810 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d7c55-5be4-44e4-ba67-0d06a80b5292 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:27:35.620290 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040d7c55-5be4-44e4-ba67-0d06a80b5292 service nova] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:27:35.632105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1479/2940] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:27:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:27:38.063336 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9988c-434f-42c8-adea-3e77c432f2a3 service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:27:38.072958 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff9988c-434f-42c8-adea-3e77c432f2a3 service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:27:38.076496 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4637858-51bd-4bf2-bdd5-0172c487102f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:27:38.077080 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4637858-51bd-4bf2-bdd5-0172c487102f service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:27:38.077387 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4637858-51bd-4bf2-bdd5-0172c487102f service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:27:38.077652 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4637858-51bd-4bf2-bdd5-0172c487102f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:27:38.087181 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4637858-51bd-4bf2-bdd5-0172c487102f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:27:38.088584 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4637858-51bd-4bf2-bdd5-0172c487102f service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:27:38.098736 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1462/2941] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:27:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:27:45.627335 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=6 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:27:48.093741 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=6 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:29:31.077986 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d7c55-5be4-44e4-ba67-0d06a80b5292 service nova] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:29:31.148751 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040d7c55-5be4-44e4-ba67-0d06a80b5292 service nova] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:29:31.154454 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d71a3-e888-4f03-a241-e5cd0a4292a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:29:31.155054 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d71a3-e888-4f03-a241-e5cd0a4292a0 service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:29:31.155334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d71a3-e888-4f03-a241-e5cd0a4292a0 service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:29:31.155606 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d71a3-e888-4f03-a241-e5cd0a4292a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:29:31.157373 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-2f6d71a3-e888-4f03-a241-e5cd0a4292a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:29:31.190596 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d71a3-e888-4f03-a241-e5cd0a4292a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:29:31.192162 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d71a3-e888-4f03-a241-e5cd0a4292a0 service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:29:31.202648 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1480/2942] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:29:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:29:41.198441 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=36 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:30:08.039729 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4637858-51bd-4bf2-bdd5-0172c487102f service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:30:08.072612 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4637858-51bd-4bf2-bdd5-0172c487102f service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:30:08.076409 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f684837-feb4-493f-9bc4-daa10f0c1248 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:30:08.077031 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f684837-feb4-493f-9bc4-daa10f0c1248 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:30:08.077338 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f684837-feb4-493f-9bc4-daa10f0c1248 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:30:08.077669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f684837-feb4-493f-9bc4-daa10f0c1248 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:30:08.079400 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-6f684837-feb4-493f-9bc4-daa10f0c1248 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:30:08.089658 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f684837-feb4-493f-9bc4-daa10f0c1248 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:30:08.091332 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f684837-feb4-493f-9bc4-daa10f0c1248 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:30:08.099235 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1463/2943] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:30:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:30:09.185315 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d71a3-e888-4f03-a241-e5cd0a4292a0 service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:30:09.194915 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d71a3-e888-4f03-a241-e5cd0a4292a0 service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:30:09.199524 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909e803-7bd8-47b7-bf13-c3696912a802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:30:09.200125 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909e803-7bd8-47b7-bf13-c3696912a802 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:30:09.200402 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909e803-7bd8-47b7-bf13-c3696912a802 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:30:09.200675 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909e803-7bd8-47b7-bf13-c3696912a802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:30:09.202401 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-b909e803-7bd8-47b7-bf13-c3696912a802 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:30:09.236951 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909e803-7bd8-47b7-bf13-c3696912a802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:30:09.238425 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b909e803-7bd8-47b7-bf13-c3696912a802 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:30:09.260636 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1481/2944] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:30:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:30:15.231215 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f684837-feb4-493f-9bc4-daa10f0c1248 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:30:15.250172 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f684837-feb4-493f-9bc4-daa10f0c1248 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:30:15.253812 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961cfa6-349f-4ba9-9498-0713dc0cb6f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:30:15.254401 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961cfa6-349f-4ba9-9498-0713dc0cb6f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:30:15.254688 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961cfa6-349f-4ba9-9498-0713dc0cb6f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:30:15.254961 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961cfa6-349f-4ba9-9498-0713dc0cb6f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:30:15.256711 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-c961cfa6-349f-4ba9-9498-0713dc0cb6f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:30:15.269933 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961cfa6-349f-4ba9-9498-0713dc0cb6f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:30:15.271347 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961cfa6-349f-4ba9-9498-0713dc0cb6f7 service neutron] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:30:15.282836 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1464/2945] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:30:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:30:16.462241 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909e803-7bd8-47b7-bf13-c3696912a802 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:30:16.487334 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909e803-7bd8-47b7-bf13-c3696912a802 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:30:16.492754 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45175-dc2a-41a0-808f-a19d98e24588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:30:16.493987 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45175-dc2a-41a0-808f-a19d98e24588 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:30:16.494434 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45175-dc2a-41a0-808f-a19d98e24588 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:30:16.494695 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45175-dc2a-41a0-808f-a19d98e24588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:30:16.496480 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-36c45175-dc2a-41a0-808f-a19d98e24588 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:30:16.511353 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c45175-dc2a-41a0-808f-a19d98e24588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:30:16.512783 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c45175-dc2a-41a0-808f-a19d98e24588 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:30:16.525970 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1482/2946] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:30:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 22 22:30:25.277932 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=21 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:30:26.519693 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=28 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:31:56.339856 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961cfa6-349f-4ba9-9498-0713dc0cb6f7 service neutron] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:31:56.351823 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961cfa6-349f-4ba9-9498-0713dc0cb6f7 service neutron] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:31:56.357785 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d23634-a42a-4c4f-a274-4d49496e9b9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd011e54e12674eaeb82c68da628ecb9d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'ResellerAdmin', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:31:56.358389 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d23634-a42a-4c4f-a274-4d49496e9b9a service nova] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:31:56.358667 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d23634-a42a-4c4f-a274-4d49496e9b9a service nova] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:31:56.358939 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d23634-a42a-4c4f-a274-4d49496e9b9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:31:56.366968 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d23634-a42a-4c4f-a274-4d49496e9b9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:31:56.368537 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d23634-a42a-4c4f-a274-4d49496e9b9a service nova] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:31:56.378253 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1465/2947] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:31:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:31:58.919319 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45175-dc2a-41a0-808f-a19d98e24588 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:31:58.928346 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45175-dc2a-41a0-808f-a19d98e24588 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:31:58.932080 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4838db-930a-408a-90ec-199e0d947ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18c956fa035f40fa88fbf662541121ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:31:58.932676 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4838db-930a-408a-90ec-199e0d947ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:31:58.932952 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4838db-930a-408a-90ec-199e0d947ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:31:58.933262 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4838db-930a-408a-90ec-199e0d947ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:31:59.002244 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4838db-930a-408a-90ec-199e0d947ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:31:59.003707 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4838db-930a-408a-90ec-199e0d947ce7 service neutron] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:31:59.022486 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1483/2948] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:31:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:32:02.049866 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d23634-a42a-4c4f-a274-4d49496e9b9a service nova] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:32:02.061553 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d23634-a42a-4c4f-a274-4d49496e9b9a service nova] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:32:02.066568 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f6502-815b-4d64-9811-f72354a3a308 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:32:02.067174 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f6502-815b-4d64-9811-f72354a3a308 service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:32:02.067454 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f6502-815b-4d64-9811-f72354a3a308 service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:32:02.067724 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f6502-815b-4d64-9811-f72354a3a308 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:32:02.069498 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fc9f6502-815b-4d64-9811-f72354a3a308 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:32:02.081982 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f6502-815b-4d64-9811-f72354a3a308 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:32:02.083590 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f6502-815b-4d64-9811-f72354a3a308 service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:32:02.095383 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1466/2949] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:32:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:32:05.188459 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4838db-930a-408a-90ec-199e0d947ce7 service neutron] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:32:05.200496 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4838db-930a-408a-90ec-199e0d947ce7 service neutron] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:32:05.206233 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc353102-47c2-4098-a78c-af49d3ba5f7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:32:05.206864 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc353102-47c2-4098-a78c-af49d3ba5f7f service placement] REQUEST_METHOD: `GET` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:32:05.207141 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc353102-47c2-4098-a78c-af49d3ba5f7f service placement] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:32:05.207407 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc353102-47c2-4098-a78c-af49d3ba5f7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:32:05.209230 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-fc353102-47c2-4098-a78c-af49d3ba5f7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:32:05.237412 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc353102-47c2-4098-a78c-af49d3ba5f7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:32:05.242316 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc353102-47c2-4098-a78c-af49d3ba5f7f service placement] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:32:05.254670 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f6502-815b-4d64-9811-f72354a3a308 service placement] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:32:05.260858 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1484/2950] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:32:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:32:05.266669 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f6502-815b-4d64-9811-f72354a3a308 service placement] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:32:05.271630 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:32:05.272286 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:32:05.272573 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:32:05.272858 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] PATH_INFO: `/v3/users/27bbd818457b4a20823fe0ec0601c312` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:32:05.274046 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27bbd818457b4a20823fe0ec0601c312)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:32:05.275978 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:32:05.327270 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27bbd818457b4a20823fe0ec0601c312'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.327937 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27bbd818457b4a20823fe0ec0601c312'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.334846 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27bbd818457b4a20823fe0ec0601c312'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.335504 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27bbd818457b4a20823fe0ec0601c312'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.351349 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27bbd818457b4a20823fe0ec0601c312'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.364392 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1467/2951] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:32:05 2022] DELETE /identity/v3/users/27bbd818457b4a20823fe0ec0601c312 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:32:05.541982 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc353102-47c2-4098-a78c-af49d3ba5f7f service placement] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:32:05.553969 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc353102-47c2-4098-a78c-af49d3ba5f7f service placement] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:32:05.618465 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:32:05.619086 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:32:05.619360 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:32:05.619630 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] PATH_INFO: `/v3/projects/285dd074069d48d2951a195965088521` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:32:05.620729 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] RBAC: Authorizing `identity:delete_project(project_id=285dd074069d48d2951a195965088521)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:32:05.622964 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:32:05.668562 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285dd074069d48d2951a195965088521', 'request_id': 'req-31d44e84-c2dc-4028-8f71-50cfda4b7413'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.674586 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285dd074069d48d2951a195965088521', 'request_id': 'req-31d44e84-c2dc-4028-8f71-50cfda4b7413'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.680122 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285dd074069d48d2951a195965088521', 'request_id': 'req-31d44e84-c2dc-4028-8f71-50cfda4b7413'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.682613 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:32:05.706229 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] The token cache is being invalidate because project 285dd074069d48d2951a195965088521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:32:05.706517 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.707142 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.708612 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1485/2952] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:32:05 2022] DELETE /identity/v3/projects/285dd074069d48d2951a195965088521 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:32:05.712636 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:32:05.714009 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:32:05.727747 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd817a14-5ee3-40ee-acd3-9cb9442883f1 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:32:05.784010 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:32:05.784623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:32:05.784898 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:32:05.785199 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] PATH_INFO: `/v3/users/f58d0f7397a34488a690a0352a2e9298` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:32:05.786310 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f58d0f7397a34488a690a0352a2e9298)` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:32:05.788196 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:32:05.834453 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f58d0f7397a34488a690a0352a2e9298'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.835104 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f58d0f7397a34488a690a0352a2e9298'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.836277 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:32:05.845478 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f58d0f7397a34488a690a0352a2e9298'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.846305 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f58d0f7397a34488a690a0352a2e9298'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.858623 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.notifications [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f58d0f7397a34488a690a0352a2e9298'} {{(pid=81628) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:05.871842 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1468/2953] 10.222.0.25 () {58 vars in 1224 bytes} [Thu Sep 22 22:32:05 2022] DELETE /identity/v3/users/f58d0f7397a34488a690a0352a2e9298 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:32:06.041680 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] Authenticating user token {{(pid=81629) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:32:06.043103 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.fernet_utils [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81629) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:32:06.062476 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d44e84-c2dc-4028-8f71-50cfda4b7413 admin admin] Validating token access rules against request {{(pid=81629) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:32:06.124338 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9a8dd819f6d742ceba5a020c855827f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f680186a14a640efb3f020f909ac5fa9', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81629) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:32:06.125090 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:32:06.125400 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:32:06.125679 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] PATH_INFO: `/v3/projects/1d561eb1ee914613a7e83b619c2bd885` {{(pid=81629) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:32:06.126807 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d561eb1ee914613a7e83b619c2bd885)` {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:32:06.128701 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] RBAC: Authorization granted {{(pid=81629) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:32:06.176105 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d561eb1ee914613a7e83b619c2bd885', 'request_id': 'req-93659a10-f464-47d9-a4f9-265aab9f8e5b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:06.182049 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d561eb1ee914613a7e83b619c2bd885', 'request_id': 'req-93659a10-f464-47d9-a4f9-265aab9f8e5b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:06.188886 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d561eb1ee914613a7e83b619c2bd885', 'request_id': 'req-93659a10-f464-47d9-a4f9-265aab9f8e5b'} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:06.203345 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] The token cache is being invalidate because project 1d561eb1ee914613a7e83b619c2bd885 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81629) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 22 22:32:06.203759 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:06.204408 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG keystone.notifications [None req-93659a10-f464-47d9-a4f9-265aab9f8e5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81629) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 22 22:32:06.205939 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: [pid: 81629|app: 0|req: 1486/2954] 10.222.0.25 () {58 vars in 1233 bytes} [Thu Sep 22 22:32:06 2022] DELETE /identity/v3/projects/1d561eb1ee914613a7e83b619c2bd885 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 22 22:32:15.868422 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=12,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:32:16.199297 dv-cinder-849143-6-iscsi devstack@keystone.service[81629]: DEBUG dbcounter [-] [81629] Writing DB stats keystone:DELETE=8,keystone:SELECT=58,keystone:INSERT=1 {{(pid=81629) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 22 22:34:33.056156 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] Authenticating user token {{(pid=81628) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 22 22:34:33.057613 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:34:33.127485 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b7b2b-ceb2-4354-a5e5-b98d3e329005 admin admin] Validating token access rules against request {{(pid=81628) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 22 22:34:33.186830 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacd17a1-7a03-4c88-9319-14f42a68c22c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43d6ba7b68884509a4e2859ceb72ef0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd7b0f546fc57464a8267c5a4f7bde1f8', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81628) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 22 22:34:33.187437 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd17a1-7a03-4c88-9319-14f42a68c22c service placement] REQUEST_METHOD: `GET` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 22 22:34:33.187709 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd17a1-7a03-4c88-9319-14f42a68c22c service placement] SCRIPT_NAME: `/identity` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 22 22:34:33.187988 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacd17a1-7a03-4c88-9319-14f42a68c22c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81628) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 22 22:34:33.189751 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.fernet_utils [None req-aacd17a1-7a03-4c88-9319-14f42a68c22c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81628) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 22 22:34:33.201316 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd17a1-7a03-4c88-9319-14f42a68c22c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 22 22:34:33.202896 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacd17a1-7a03-4c88-9319-14f42a68c22c service placement] RBAC: Authorization granted {{(pid=81628) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 22 22:34:33.213626 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: [pid: 81628|app: 0|req: 1469/2955] 10.222.0.25 () {60 vars in 1365 bytes} [Thu Sep 22 22:34:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 22 22:34:43.208475 dv-cinder-849143-6-iscsi devstack@keystone.service[81628]: DEBUG dbcounter [-] [81628] Writing DB stats keystone:SELECT=58 {{(pid=81628) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}